Systems and methods for manipulating light from ambient light sources

ABSTRACT

An optical device includes variable optical material that alters at least one of: incident ambient light, spectral content of incident ambient light or direction of incident ambient light through the optical device in response to a stimulus provided by the device. The device can sense intensity and/or spectral characteristics of ambient light and provide appropriate stimulus to various portions of the optical device to activate the variable optical material and alter at least one of: incident ambient light, spectral content of incident ambient light or direction of incident ambient light.

PRIORITY CLAIM

This application claims the priority benefit of U.S. Provisional PatentApplication No. 62/438,325 filed on Dec. 22, 2016, which is incorporatedby reference herein in its entirety.

INCORPORATION BY REFERENCE

This application is also related to U.S. patent application Ser. No.15/841,043, filed on Dec. 13, 2017, which is incorporated by referenceherein in its entirety.

BACKGROUND Field

The present disclosure relates to optical devices, including virtualreality and augmented reality imaging and visualization systems.

Description of the Related Art

Modern computing and display technologies have facilitated thedevelopment of systems for so called “virtual reality” or “augmentedreality” experiences, wherein digitally reproduced images or portionsthereof are presented to a user in a manner wherein they seem to be, ormay be perceived as, real. A virtual reality, or “VR”, scenariotypically involves presentation of digital or virtual image informationwithout transparency to other actual real-world visual input; anaugmented reality, or “AR”, scenario typically involves presentation ofdigital or virtual image information as an augmentation to visualizationof the actual world around the user. A mixed reality, or “MR”, scenariois a type of AR scenario and typically involves virtual objects that areintegrated into, and responsive to, the natural world. For example, inan MR scenario, AR image content may be blocked by or otherwise beperceived as interacting with objects in the real world.

Referring to FIG. 1, an augmented reality scene 10 is depicted wherein auser of an AR technology sees a real-world park-like setting 20featuring people, trees, buildings in the background, and a concreteplatform 30. In addition to these items, the user of the AR technologyalso perceives that he “sees” “virtual content” such as a robot statue40 standing upon the real-world platform 30, and a cartoon-like avatarcharacter 50 flying by which seems to be a personification of a bumblebee, even though these elements 40, 50 do not exist in the real world.Because the human visual perception system is complex, it is challengingto produce an AR technology that facilitates a comfortable,natural-feeling, rich presentation of virtual image elements amongstother virtual or real-world imagery elements.

Systems and methods disclosed herein address various challenges relatedto AR and VR technology.

SUMMARY

The systems, methods and devices of the disclosure each have severalinnovative aspects, no single one of which is solely responsible for thedesirable attributes disclosed herein.

Details of one or more embodiments of the subject matter described inthis specification are set forth in the accompanying drawings and thedescription below. Other features, aspects, and advantages will becomeapparent from the description, the drawings, and the claims. Note thatthe relative dimensions of the following figures may not be drawn toscale.

Various examples of optical devices comprising a variable opticalmaterial that undergoes a physical and/or a chemical change in responseto a stimulus are described herein such as the examples enumeratedbelow:

EXAMPLE 1

A user-wearable display device comprising: a frame configured to mounton the user; an augmented reality display attached to the frame andconfigured to direct images to an eye of the user; a sensor configuredto obtain information about ambient light condition in an environmentsurrounding the user; a variable optical material that undergoes aphysical and/or a chemical change in response to a stimulus; a sourceconfigured to provide the stimulus; and processing electronicsconfigured to: trigger the source to provide the stimulus to thevariable optical material to effect a physical and/or a chemical changein the material based on the information obtained by the sensor suchthat at least one of intensity of ambient light, spectral content ofambient light or direction of ambient light is changed.

EXAMPLE 2

The user-wearable device of Example 1, wherein the augmented realitydisplay comprises a waveguide configured to: allow a view of theenvironment surrounding the user through the waveguide; and form imagesby directing light out of the waveguide and into an eye of the user.

EXAMPLE 3

The user-wearable device of Examples 1-2, wherein the waveguide is partof a stack of waveguides, wherein each waveguide of the stack isconfigured to output light with different amounts of divergence incomparison to one or more other waveguides of the stack of waveguides.

EXAMPLE 4

The user-wearable device of Examples 1-3, wherein the sensor comprisesat least one of a light sensor, an image capture device, a globalpositioning sub-system, or an environmental sensor.

EXAMPLE 5

The user-wearable device of Examples 1-4, further comprising an imagecapture device configured to track movement of eyes of the user.

EXAMPLE 6

The user-wearable device of Examples 1-5, further comprising a lightsource configured to generate a projection beam based on data associatedwith the images directed to the eye of the user.

EXAMPLE 7

The user-wearable device of Examples 1-6, wherein the source comprisesan optical source configured to direct visible or invisible light to oneor more portions of the display.

EXAMPLE 8

The user-wearable device of Examples 1-6, wherein the source comprisesan electrical source configured to provide an electrical signal to oneor more portions of the display.

EXAMPLE 9

The user-wearable device of Examples 1-6, wherein the source comprises athermal source configured to provide a thermal radiation to one or moreportions of the display.

EXAMPLE 10

The user-wearable device of Examples 1-6, wherein the source comprises asonic/ultrasonic system configured to provide sonic/ultrasonic energy toone or more portions of the display.

EXAMPLE 11

The user-wearable device of Examples 1-10, wherein the variable opticalmaterial is embedded in a surface of the display.

EXAMPLE 12

The user-wearable device of Examples 1-10, wherein the variable opticalmaterial is disposed over a surface of the display.

EXAMPLE 13

The user-wearable device of Examples 1-12, wherein the variable opticalmaterial includes organic or inorganic compounds.

EXAMPLE 14

The user-wearable device of Examples 1-13, wherein the variable opticalmaterial comprises electroactive proteins.

EXAMPLE 15

The user-wearable device of Examples 1-14, wherein the variable opticalmaterial comprises molecules that exhibit a change is size or shape inresponse to the stimulus.

EXAMPLE 16

The user-wearable device of Examples 1-15, wherein the variable opticalmaterial comprises molecules that move, rotate, twist or shift inresponse to the stimulus.

EXAMPLE 17

The user-wearable device of Examples 1-16, wherein the variable opticalmaterial comprises molecules that move together and/or adhere togetherin response to the stimulus.

EXAMPLE 18

The user-wearable device of Examples 1-16, wherein the variable lightoptical material comprises molecules that move away from each other inresponse to the stimulus.

EXAMPLE 19

The user-wearable device of Examples 1-18, wherein the variable opticalmaterial comprises molecules that form nanostructures in response to thestimulus.

EXAMPLE 20

The user-wearable device of Examples 1-19, wherein the display comprisesa first ocular region corresponding to a first eye of the user and asecond ocular region corresponding to a second eye of the user, andwherein the processing electronics is configured to trigger the sourceto provide the stimulus to a portion of the display to effect a physicaland/or a chemical change in the variable optical material based on theinformation obtained by the sensor such that at least one of intensityof ambient light, spectral content of ambient light or direction ofambient light is changed through the first ocular region as a result ofstimulus from a source triggered by the processing electronics.

EXAMPLE 21

The user-wearable device of Examples 1-19, wherein the display comprisesa first ocular region corresponding to a first eye of the user and asecond ocular region corresponding to a second eye of the user, andwherein the processing electronics is configured to trigger the sourceto provide the stimulus to a portion of the display to effect a physicaland/or a chemical change in the material based on the informationobtained by the sensor such that at least one of intensity of ambientlight, spectral content of ambient light or direction of ambient lightthrough the first ocular region is changed differently as compared tointensity of ambient light, spectral content of ambient light ordirection of ambient light through the second ocular region.

EXAMPLE 22

The user-wearable device of Examples 1-19, wherein the processingelectronics is configured to trigger the source to provide the stimulusto the display to effect a physical and/or a chemical change in thematerial based on the information obtained by the sensor such thatattenuation of intensity of ambient light transmitted through a firstportion of the display is greater than attenuation of intensity ofambient light transmitted through a second portion of the display.

EXAMPLE 23

The user-wearable device of Examples 22, wherein the intensity ofambient light incident on the first portion of the display is greaterthan intensity of ambient light incident on the second portion of thedisplay.

EXAMPLE 24

The user-wearable device of Examples 22 or 23, wherein the processingelectronics is configured to trigger the source to provide the stimulusto the display to effect a physical and/or a chemical change in thematerial based on the information obtained by the sensor such that theintensity of ambient light transmitted through the second portion of thedisplay is reduced.

EXAMPLE 25

The user-wearable device of Examples 1-19, wherein the display comprisesa first ocular region corresponding to a first eye of the user and asecond ocular region corresponding to a second eye of the user, andwherein the processing electronics is configured to trigger the sourceto provide the stimulus to the display to effect a physical and/or achemical change in the material based on the information obtained by thesensor such that intensity of ambient light transmitted through aportion of the first ocular region is reduced.

EXAMPLE 26

The user-wearable device of Examples 1-19, wherein the processingelectronics is configured to trigger the source to provide the stimulusto the display to effect a physical and/or a chemical change in thematerial based on the information obtained by the sensor such that thespectrum of ambient light transmitted through a first portion of thedisplay is different than the spectrum of ambient light transmittedthrough a second portion of the display.

EXAMPLE 27

The user-wearable device of Examples 1-19, wherein the display comprisesa first lens corresponding to a first eye of the user and a second lenscorresponding to a second eye of the user, and wherein the processingelectronics is configured to trigger the source to provide the stimulusto the display to effect a physical and/or a chemical change in thevariable optical material associated with the first lens based on theinformation obtained by the sensor such that intensity of ambient lighttransmitted through only the first lens is reduced as a result ofstimulus from a source triggered by the processing electronics.

EXAMPLE 28

The user-wearable device of Examples 1-19, wherein the display comprisesa first lens corresponding to a first eye of the user and a second lenscorresponding to a second eye of the user, and wherein the processingelectronics is configured to trigger the source to provide the stimulusto the display to effect a physical and/or a chemical change in thevariable optical material associated with the first lens based on theinformation obtained by the sensor such that intensity of ambient lighttransmitted through a portion of the first lens is reduced by an amountgreater than another portion of the first lens.

EXAMPLE 29

The user-wearable device of Example 28, wherein the processingelectronics is configured to trigger the source to provide the stimulusto the display to effect a physical and/or a chemical change in thevariable optical material associated with the second lens based on theinformation obtained by the sensor such that intensity of ambient lighttransmitted through a portion of the second lens is reduced.

EXAMPLE 30

The user-wearable device of Examples 1-19, wherein the display comprisesa first lens corresponding to a first eye of the user and a second lenscorresponding to a second eye of the user, and wherein the processingelectronics is configured to trigger the source to provide the stimulusto the display to effect a physical and/or a chemical change in thevariable optical material associated with the first lens based on theinformation obtained by the sensor such that intensity of ambient lighttransmitted through the first lens is attenuated more than through thesecond lens.

EXAMPLE 31

The user-wearable device of Example 30, wherein the processingelectronics is configured to trigger the source to provide the stimulusto the display to effect a physical and/or a chemical change in thevariable optical material associated with the second lens based on theinformation obtained by the sensor such that intensity of ambient lighttransmitted through the second lens is reduced.

EXAMPLE 32

The user-wearable device of Examples 1-19, wherein the display comprisesa first lens corresponding to a first eye of the user and a second lenscorresponding to a second eye of the user, and wherein the processingelectronics is configured to trigger the source to provide the stimulusto the display to effect a physical and/or a chemical change in variableoptical material associated with the first or second lens based on theinformation obtained by the sensor such that spectrum of ambient lighttransmitted through the first and second lenses is different.

EXAMPLE 33

The user-wearable device of Examples 1-19, wherein the display comprisesa first lens corresponding to a first eye of the user and a second lenscorresponding to a second eye of the user, and wherein the processingelectronics is configured to trigger the source to provide the stimulusto the display to effect a physical and/or a chemical change in thevariable optical material associated with the first or second lens basedon the information obtained by the sensor such that the spectrum ofambient light transmitted through a portion of the first lenses isdifferent than another portion of the first lens.

EXAMPLE 34

The user-wearable device of Example 33, wherein the display comprises afirst lens corresponding to a first eye of the user and a second lenscorresponding to a second eye of the user, and wherein the processingelectronics is configured to trigger the source to provide the stimulusto the display to effect a physical and/or a chemical change in thevariable optical material associated with the first or second lens basedon the information obtained by the sensor such that the spectrum ofambient light transmitted through a portion of the first lenses isdifferent than another portion of the second lens.

EXAMPLE 35

The user-wearable device of Examples 1-19, wherein an object as seen bythe wearer's eye through the display appears to be aligned with at leastone portion of the display, and wherein the processing electronics isconfigured to cause the source to provide the stimulus to the at leastone portion of the display for which the object appears to be aligned toeffect a physical and/or a chemical change in the variable opticalmaterial such that at least one of intensity of light from said object,spectral content of said light from said object or direction of saidlight from said object is changed.

EXAMPLE 36

The user-wearable device of Example 35, wherein the processingelectronics is configured to determine the at least one portion of thedisplay for which the object appears to be aligned based on the movementof the user's head as tracked by said sensor.

EXAMPLE 37

The user-wearable device of any of Example 35-36, wherein the processingelectronics is configured to cause the source to provide the stimulus tothe at least one portion of the display to effect a physical and/or achemical change in the variable optical material such that the intensityof ambient light reduced.

EXAMPLE 38

The user-wearable device of any of the Examples above, furthercomprising a head pose sensor.

EXAMPLE 39

The user-wearable device of any of the Examples above, furtherconfigured to adjust the location of the at least one portion of thedisplay through which at least one of intensity of ambient light,spectral content of ambient light or direction of ambient light ischanged based on feedback from the user.

EXAMPLE 40

The user-wearable device of any of the Examples above, furtherconfigured to adjust the size of the at least one portion of the displaythrough which at least one of intensity of ambient light, spectralcontent of ambient light or direction of ambient light is changed basedon feedback from the user.

EXAMPLE 41

The user-wearable device of any of the Examples above, furtherconfigured to adjust the amount by which at least one of intensity ofambient light, spectral content of ambient light or direction of ambientlight is changed based on feedback from the user.

EXAMPLE 42

A method of manipulating light transmitted through a user-wearabledisplay device comprising a display surface including a variable opticalmaterial that varies at least one of intensity of ambient light,spectral content of ambient light or direction of ambient lighttransmitted through the display surface in response to a stimulus, themethod comprising: obtaining measurement about ambient light conditionin an environment surrounding the user using a sensor; determiningintensity of light incident on a first location associated with a firstportion of the display surface and a second location associated with asecond portion of the display surface, said first location closer tosaid first portion of the display surface than said second portion, saidsecond location closer to said second portion of the display surfacethan said first portion; controlling a source to provide a firststimulus to the first portion of the display surface to effect aphysical and/or chemical change in the material such that at least oneof intensity of ambient light, spectral content of ambient light ordirection of ambient light incident on the first portion is changed by afirst amount; and controlling the source to provide a second stimulus tothe second portion of the display surface to effect a physical and/orchemical change in the material such that at least one of intensity ofambient light, spectral content of ambient light or direction of ambientlight incident on the second portion is changed by a second amount.

EXAMPLE 43

The method of Example 42, wherein the first amount is different than thesecond amount.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a user's view of augmented reality (AR) through an ARdevice.

FIGS. 2A and 2B illustrate embodiments of a wearable display system.

FIG. 3 illustrates a conventional display system for simulatingthree-dimensional imagery for a user.

FIG. 4 illustrates aspects of an approach for simulatingthree-dimensional imagery using multiple depth planes.

FIGS. 5A-5C illustrate relationships between radius of curvature andfocal radius.

FIG. 6 illustrates an example of a waveguide stack for outputting imageinformation to a user.

FIG. 7 illustrates an example of exit beams outputted by a waveguide.

FIG. 8 illustrates an example of a stacked waveguide assembly in whicheach depth plane includes images formed using multiple differentcomponent colors.

FIG. 9A illustrates a cross-sectional side view of an example of a setof stacked waveguides that each includes an in-coupling optical element.

FIG. 9B illustrates a perspective view of an example of the plurality ofstacked waveguides of FIG. 9A.

FIG. 9C illustrates a top-down plan view of an example of the pluralityof stacked waveguides of FIGS. 9A and 9B.

FIG. 10 illustrates a scene including one or more sources of ambientlight.

FIG. 11 is a flowchart illustrating a method of varying transmission oflight through a display lens.

FIG. 12A is a side view of a display lens including a portion withreduced ambient light transmission. FIG. 12B is a front view of thedisplay lens illustrated in FIG. 12A as seen from a side opposite theeye side. FIG. 12C is a top view of the display lens illustrated in FIG.12A.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

The embodiments contemplated herein include a wearable display device(e.g., an augmented reality and/or virtual reality eyewear) comprisingat least one variable optical material that can vary at least one of:the intensity of ambient light transmitted through the display device,spectral content of ambient light transmitted through the displaydevice, or the optical path of the ambient light transmitted through thedisplay device (e.g., by diffraction or by changing the refractive indexof the variable optical element) in response to an external stimulus(e.g., an optical stimulus, an electrical stimulus, a thermal stimulus,an ultrasonic/sonic stimulus, a radiation pressure, etc.). In variousembodiments, the at least one variable optical material can beconfigured to attenuate the intensity of the ambient light in one ormore wavelength ranges. In some embodiments, the at least one variableoptical material can be configured to reflect, refract, scatter,diffract or absorb incoming light. The wearable display device takesadvantage of the physical changes/chemical changes that are broughtabout in the at least one variable optical material by the externalstimulus. As a result of the external stimulus, the at least onevariable optical material can vary at least one of the intensity ofambient light transmitted through the display device, spectral contentof ambient light transmitted through the display device, or the opticalpath of the ambient light transmitted through the display devicedepending on the intensity and/or spectral characteristics of theincoming light to improve user experience. Various studies can beperformed to characterize the light altering characteristics of thevariable optical material. Different studies can also be performed tocharacterize the type of light alteration that will result in a desireduser experience for different types of ambient light sources. Feedbackfrom the various studies can be taken into consideration to determinewhich regions of the display device should have altered lighttransmission and the amount of light alteration that would provide thedesired user experience.

In some embodiments, the at least one variable optical material can beembedded in a display surface of the display device. In some otherembodiments, the at least one variable optical material can be includedin an accessory component that can be disposed over the display device.The at least one variable optical material can include photosensitive,electro-active and/or radiosensitive materials. In some embodiments, theat least one variable optical material can comprise organic or inorganiccompounds. In some embodiments, the at least one variable opticalmaterial can comprise photosensitive materials, such as, for example,silver-based compounds (e.g., silver chloride or silver halide). In someother embodiments, the at least one variable optical material cancomprise organic compounds such as oxazines and/or napthopyrans. In someembodiments, the at least one variable optical material can comprise oneor more layers of molecules.

The at least one variable optical material can be activated by anoptical stimulus provided from a source of illumination, for example, onthe eyewear or integrated with the eyewear. The source of illuminationcan be monochromatic or polychromatic. In various embodiments, thesource of illumination can include a LED, a scanning fiber projector, anultraviolet source of light or a source configured to provide anelectron beam. The source of illumination can be controlled byelectrical or mechanical devices. For example, in some embodiments, thesource of illumination can be controlled by a movable shutter or avariable filter. As another example, the source of illumination can beelectrically controlled by a processor.

The processor is configured to trigger the device that provides optical,electrical, thermal and/or sonic/ultrasonic stimulus based oninformation obtained from one or more sensors (e.g., a light sensor, oneor more cameras, eye-tracking cameras, position sensing devices, posesensing devices, environmental sensors configured to detect temperature,global positioning system sub-assemblies, accelerometers, color sensors,etc.). For example, the processor can be configured to turn on or turnoff, activate or deactivate, or otherwise control the device thatprovides optical, electrical, thermal and/or sonic/ultrasonic stimulusthat would activate or control the at least one variable material indifferent portions of the display device to change at least one of: theintensity of ambient light transmitted through the display device,spectral content of ambient light transmitted through the displaydevice, or the optical path of the ambient light transmitted through thedisplay device based on information obtained from the one or moresensors.

In response to the stimulus, the at least one variable optical materialcan undergo a physical and/or a chemical change. For example, themolecules of the at least one variable optical material can undergo achange in size (e.g., shrink or enlarge) in response to the stimulus. Asanother example, the molecules of the at least one variable opticalmaterial can undergo a change in shape in response to the stimulus. Asyet another example, density of the molecules of the at least onevariable optical material can change in response to the stimulus. As aresult, the stimulus may change at least one of: the intensity ofambient light transmitted through the display device, spectral contentof ambient light transmitted through the display device, or the opticalpath of the ambient light transmitted through the display device.

In various embodiments, the molecules of the at least one variableoptical material may be configured to move, shift, rotate, twist orotherwise change or respond upon providing the stimulus. The movement,shift, rotation or twisting of molecules of the at least one variableoptical material may be configured to be random in some embodiments.However, in some other embodiments, the movement, shift, rotation ortwisting of molecules of the at least one variable optical material maybe configured to be along a specific direction. In some embodiments, thespeed with which the molecules of the at least one variable opticalmaterial are moved, shifted, rotated or twisted can be varied bychanging a characteristic of the stimulus provided. In variousembodiments, the molecules of the at least one variable optical materialcan be moved closer together in response to the stimulus. In some otherembodiments, the molecules of the at least one variable optical materialcan be moved farther apart from each other in response to the stimulus.In some embodiments, the molecules of the at least one variable opticalmaterial can be configured to form nanostructures in response to thestimulus.

The physical and/or chemical change of the molecules of the at least onevariable optical material can be brought about by controlling acharacteristic of the stimulus. For example, when the stimulus isoptical, the physical and/or chemical change of the molecules of the atleast one variable optical material can be brought about by controllingthe wavelength and/or intensity of the optical stimulus. As anotherexample, when the stimulus is electrical, the physical and/or chemicalchange of the molecules of the at least one variable optical materialcan be brought about by controlling the voltage and/or current of theelectrical stimulus. In various embodiments, the physical and/orchemical change of the molecules of the at least one variable opticalmaterial can be controlled by modulating the source that provides thestimulus. In some embodiments, the physical and/or chemical change ofthe molecules of the at least one variable optical material can bereversible such that when the stimulus is removed, the molecules of theat least one variable optical material revert back to their originalstate. In such embodiments, the stimulus is constantly provided tomaintain the altered state of the molecules of the at least one variableoptical material. In some other embodiments the physical and/or chemicalchange of the molecules of the at least one variable optical materialcan be maintained in the absence of the stimulus until deactivationenergy is provided to revert the molecules of the at least one variableoptical material to their original state. In such embodiments, thestimulus can be provided for a short duration of time to initiate thealteration of the molecules of the at least one variable opticalmaterial.

Various embodiments of the wearable display device are configured to mapobjects in the real world surrounding the user, including objects thatare visible to the user through the display device, using a variety ofsensor assemblies and/or imaging apparatus. In various embodiments, theinformation obtained from the variety of sensor assemblies and/orimaging apparatus can be used to create a database including, forexample, the position of various objects in the real world with respectto the display device and/or the user's head/eyes and potentially othercharacteristics of the objects such as their size, shape, and/or howbright the objects appear. The database can be updated and/or provideupdated information in real time or in near real time as the objects inthe surrounding real world appear to move with respect to the displaydevice and/or the user's head/eyes as the user moves his/her head and/orbody. The database can be updated and/or provide updated information inreal time or in near real time regarding position of new objects fromthe surrounding real world that come into the user's field of view asthe user moves his/her head. The display device can be configured and/orused to locate and identify different ambient light sources in the realworld visible to the user through the display device. The differentambient light sources may appear to be aligned with different portionsof the viewable surface of the display device. These objects may produceglare. Accordingly, the display device can be configured to change,alter, adjust or manipulate at least one of: the intensity of ambientlight, the optical path of the ambient light, or the spectral content ofambient light transmitted through different portions of the viewablesurface of the display device with which the different ambient lightsources appear to be aligned in order to reduce glare.

Various embodiments of the wearable display device are configured toattenuate incoming ambient light incident on various portions of thedisplay surface. Accordingly, the amount of variation of at least oneof: the intensity of ambient light transmitted through the displaydevice, spectral content of ambient light transmitted through thedisplay device, or the optical path of the ambient light transmittedthrough the display device can vary across the surface of the displaydevice and need not be uniform. This can be advantageous in maintaininguser experience when one portion of the display surface introduces moreglare than another portion. For example, when a user is viewing a scenewith the sun or a bright light in the background, then incoming lighttransmitted through a portion of the display device that is aligned withthe sun or bright light can be attenuated by a larger amount thanintensity of incoming light transmitted through other portions of thedisplay device. Additionally, when a user is viewing the display devicenear a window or using a desk light, then incoming light transmittedthrough a portion of the display device near the window or the desklight can be attenuated by a larger amount than intensity of incominglight transmitted through a portion of the display device farther fromthe window or the desk light, since the portion of the display devicenear the window or the desk light may have more glare.

Reference will now be made to the figures, in which like referencenumerals refer to like parts throughout. It will be appreciated thatembodiments disclosed herein include optical systems, including displaysystems, generally. In some embodiments, the display systems arewearable, which may advantageously provide a more immersive VR or ARexperience. For example, displays containing one or more waveguides(e.g., a stack of waveguides) may be configured to be worn positioned infront of the eyes of a user, or viewer. In some embodiments, two stacksof waveguides, one for each eye of a viewer, may be utilized to providedifferent images to each eye.

Example Display Systems

FIG. 2A illustrates an example of wearable display system 60. Thedisplay system 60 includes a display 70, and various mechanical andelectronic modules and systems to support the functioning of thatdisplay 70. The display 70 may be coupled to a frame 80, which iswearable by a display system user or viewer 90 and which is configuredto position the display 70 in front of the eyes of the user 90. Thedisplay 70 may be considered eyewear in some embodiments. In someembodiments, a speaker 100 is coupled to the frame 80 and configured tobe positioned adjacent the ear canal of the user 90 (in someembodiments, another speaker, not shown, is positioned adjacent theother ear canal of the user to provide stereo/shapeable sound control).In some embodiments, the display system may also include one or moremicrophones 110 or other devices to detect sound. In some embodiments,the microphone is configured to allow the user to provide inputs orcommands to the system 60 (e.g., the selection of voice menu commands,natural language questions, etc.), and/or may allow audio communicationwith other persons (e.g., with other users of similar display systems.The microphone may further be configured as a peripheral sensor tocollect audio data (e.g., sounds from the user and/or environment). Insome embodiments, the display system may also include a peripheralsensor 120 a, which may be separate from the frame 80 and attached tothe body of the user 90 (e.g., on the head, torso, an extremity, etc. ofthe user 90). The peripheral sensor 120 a may be configured to acquiredata characterizing the physiological state of the user 90 in someembodiments. For example, the sensor 120 a may be an electrode.

With continued reference to FIG. 2A, the display 70 is operativelycoupled by communications link 130, such as by a wired lead or wirelessconnectivity, to a local data processing module 140 which may be mountedin a variety of configurations, such as fixedly attached to the frame80, fixedly attached to a helmet or hat worn by the user, embedded inheadphones, or otherwise removably attached to the user 90 (e.g., in abackpack-style configuration, in a belt-coupling style configuration).Similarly, the sensor 120 a may be operatively coupled by communicationslink 120 b, e.g., a wired lead or wireless connectivity, to the localprocessor and data module 140. The local processing and data module 140may comprise a hardware processor, as well as digital memory, such asnon-volatile memory (e.g., flash memory or hard disk drives), both ofwhich may be utilized to assist in the processing, caching, and storageof data. The data include data a) captured from sensors (which may be,e.g., operatively coupled to the frame 80 or otherwise attached to theuser 90), such as image capture devices (such as cameras), microphones,inertial measurement units, accelerometers, compasses, GPS units, radiodevices, gyros, and/or other sensors disclosed herein; and/or b)acquired and/or processed using remote processing module 150 and/orremote data repository 160 (including data relating to virtual content),possibly for passage to the display 70 after such processing orretrieval. The local processing and data module 140 may be operativelycoupled by communication links 170, 180, such as via a wired or wirelesscommunication links, to the remote processing module 150 and remote datarepository 160 such that these remote modules 150, 160 are operativelycoupled to each other and available as resources to the local processingand data module 140. In some embodiments, the local processing and datamodule 140 may include one or more of the image capture devices,microphones, inertial measurement units, accelerometers, compasses, GPSunits, radio devices, and/or gyros. In some other embodiments, one ormore of these sensors may be attached to the frame 80, or may bestandalone structures that communicate with the local processing anddata module 140 by wired or wireless communication pathways.

With continued reference to FIG. 2A, in some embodiments, the remoteprocessing module 150 may comprise one or more processors configured toanalyze and process data and/or image information. In some embodiments,the remote data repository 160 may comprise a digital data storagefacility, which may be available through the internet or othernetworking configuration in a “cloud” resource configuration. In someembodiments, the remote data repository 160 may include one or moreremote servers, which provide information, e.g., information forgenerating augmented reality content, to the local processing and datamodule 140 and/or the remote processing module 150. In some embodiments,all data is stored and all computations are performed in the localprocessing and data module, allowing fully autonomous use from a remotemodule.

Various embodiments of the display system 60 can include one or morecomponents (e.g., cameras, light sensors, color sensors, temperaturesensors, motion detectors, accelerometers, gyroscopes, globalpositioning sub-systems, etc.) that are configured to sense theenvironment surrounding the user 90. The one or more components includedin the display system 60 can also be configured to monitor the positionof the head and/or track eye movements of the user 90. For example, theone or more components included in the display system 60 can beconfigured to determine constriction of the pupil in response to brightlight, enlargement of the pupil in response to low light, blinkresponse, etc. As another example, the one or more components includedin the display system 60 can be configured to monitor and/or trackmovement of the user's head. In some embodiments, the one or morecomponents included in the display system 60 can be configured tomonitor and/or track position of real world objects (e.g., trees, sun,ambient light sources, etc.) with respect to the user's eyes as theuser's head moves.

FIG. 2B illustrates some of the components included in an embodiment ofthe display system 60. Other embodiments may have additional or fewercomponents depending on the application for which the system is used.Nevertheless, FIG. 2B provides a basic idea of some of the variouscomponents that can be included in the display system 60 that areconfigured to sense the environment. In the embodiment illustrated inFIG. 2B, the display device 70 comprises a display lens 106 that may bemounted to a user's head or eyes by the frame 80. The display lens 106may be configured to propagate projected light 124 from one or morelight projection systems 118 into the eyes 122. The display lens 106 canalso be configured to allow for transmission of at least some light fromthe local environment surrounding the user 90. In various embodiments ofthe display system 60 configured as an augmented reality device, theprojected light 124 can include virtual content that may be superimposedon the real world content viewed by the user's eyes 122.

The display system can include one or more outward facing cameras 112that are configured to image the environment around the user 90. In someembodiments, the cameras 112 can comprise wide-field-of-view machinevision cameras. In some embodiments, the cameras 112 can be dual capturevisible light/non-visible (e.g., infrared) light cameras. The cameras112 can be integrated with the frame 80 as depicted in FIG. 2B. However,in some embodiments, the cameras 112 can be positioned elsewhere. Forexample, the cameras 112 can be configured to be attached to the head,arms, neck or some other parts of the body of the user 90. In variousembodiments, the cameras 112 need not be attached to the user 90 butinstead, can be positioned beside the user.

With continued reference to FIG. 2B, the display system 60 can includeone or more inward facing cameras 114 that can be configured to monitorthe user's eyes 122. In various embodiments, the inward facing cameras114 can be paired with infrared light sources (such as light emittingdiodes “LED”s), which are configured to track the eyes 122 of the user90. The system 60 can further comprise one or more light sensors 128that are configured to sense ambient light. For example, the one or morelight sensors 128 can be configured to sense at least one of intensity,wavelength or color temperature or range of the ambient light. Invarious embodiments, the light sensor 128 can comprise siliconphotodetectors, phototransistors, photodiodes, LCD sensors, sensors thatuse resistance properties to detect changes in the intensity/spectralcharacteristic of light, infrared (IR) light sensors, etc. The system 60can further comprise a sensor assembly 126, which may comprise one ormore X, Y, and Z axis accelerometers as well as a magnetic compass andone or more X, Y, and Z axis gyros, preferably providing data at arelatively high frequency, such as 200 Hz. In some embodiments, thesensor assembly 126 can comprise a global positioning satellite (GPS)subsystem to provide information about the user's environment.

The local processing and data module 140 and/or the remote processingmodule 150 may comprise a processor such as an ASIC (applicationspecific integrated circuit), FPGA (field programmable gate array),and/or ARM processor (advanced reduced-instruction-set machine), whichmay be configured to calculate real or near-real time user head posefrom the information obtained by the inward facing cameras 114, theoutward facing cameras 112, light sensor 128, and/or the sensor assembly126. The processor can be configured to provide information about theuser's environment from the information obtained by the inward facingcameras 114, the outward facing cameras 112, the light sensor 128 and/orthe sensor assembly 126. In various embodiments, using the informationobtained from the outward facing cameras 112, the light sensor 128and/or the sensor assembly 126, the display system 60 can be configuredto determine the ambient light conditions. For example, the informationobtained from the outward facing cameras 112, the light sensor 128and/or the sensor assembly 126 can be processed using one or moreelectronic processors of the local processing and data module 140 and/orthe remote processing module 150 to determine whether the ambient lightis diffused. If the ambient light is not diffused, then the system 60can use the information obtained from the outward facing cameras 112,the light sensor 128 and/or the sensor assembly 126 to determine thedirection from which ambient light is incident on the display 70. Thesystem 60 can be configured to determine the type of illuminant thatprovides the ambient light. For example, the system 60 can be configuredto determine whether the illuminant is sunlight or light from anartificial light source. As another example, the system 60 can beconfigured to determine the spectral composition and/or the intensity ofambient light from the information obtained from the outward facingcameras 112, the light sensor 128 and/or the sensor assembly 126.

As discussed above, the inward facing cameras 114 may be utilized totrack the eyes. Accordingly, the information provided by the inwardfacing cameras 114 can be used to determine the object at which or thedirection along which the user is looking, as well as the depth at whichthe user's eyes are focusing. The information provided by the inwardfacing cameras 114 can also be used to determine the ambient lightcondition. For example, the information obtained by the light sensor128, the sensor assembly 126, the outward facing cameras 112 andpossibly one or more head pose sensors can be combined with theinformation provided by the inward facing cameras 114 regarding the sizeof the pupil of the user's eyes 122 to determine the pose of the user'shead (and/or eyes) and locate and identify different ambient lightsources in the real world visible to the user through the displaydevice. The system 60 can be configured to determine the direction alongwhich ambient light is incident, the intensity of ambient light and/orthe spectral characteristics of the ambient light that is incident onthe display 70. The information obtained by the light sensor 128, thesensor assembly 126, the outward facing cameras 112, and possibly one ormore head pose sensors, regarding the location of object as well aspossibly the pose of the user's head can be combined with theinformation provided by the inward facing cameras 114 regarding the sizeof the pupil of the user's eyes 122 and possibly the direction that theuser's eyes are pointing, to identify portions of the display 70 thatcoincide, are aligned with and/or overlap with the ambient light sourcesin the view of the real world visible to the user. The information fromthe light sensor 128, the sensor assembly 126, the outward facingcameras 112 and/or inward facing cameras 114 may be utilized inconjunction with data possibly from an associated cloud computingresource, to map the local world and object, features or characteristicsthereof and the position of the objects and features of the local worldwith respect to the eyes of the user.

In various embodiments as discussed below, the display lens 106 caninclude a variable optical component having at least one material thatcan be configured to vary at least one of: the intensity of ambientlight transmitted through at least a portion of the display lens 106,spectral content of ambient light transmitted through at least a portionof the display lens 106, or the optical path of the ambient lighttransmitted through at least a portion of the display lens 106 inresponse to a stimulus provided by one or more components of the displaysystem 60 to improve user experience. For example, if the display system60 determines based on the information obtained from the light sensor128, the sensor assembly 126, the outward facing cameras 112 and/orinward facing cameras 114 that the ambient light conditions on a portionof the display lens 106 are bright or that a bright object is in thefield of view of the user and is aligned with a portion of the display,then the display system 60 can be configured to provide a stimulus(e.g., thermal, sonic/ultrasonic, optical or electrical stimulus) thatcan change at least one of: the intensity of ambient light transmittedthrough that portion of the display lens 106, spectral content ofambient light transmitted through that portion of the display lens 106,or the optical path of the ambient light transmitted through thatportion of the display lens 106 to reduce intensity of ambient lighttransmitted through that portion of the display lens 106 and/or from thebright object and improve visual experience.

Accordingly, various embodiments of the display system 60 can comprise alight emitting module 134 that is configured to emit ultraviolet,infrared and/or visible light to provide an optical stimulus to at leasta portion of the display lens 106; an electrical system 132 that canprovide an electrical stimulus to at least a portion of the display lens106; a thermal source 136 that can provide a thermal stimulus to atleast a portion of the display lens 106; and/or a sonic/ultrasonictransducer 138 to provide sonic and/or ultrasonic stimulus to at least aportion of the display lens 106. The optical stimulus provided by thelight emitting module 134 can include a directed narrow beam ofinvisible and/or visible light that is incident on the portion of thedisplay lens 106 that is configured to have reduced light transmission.In various embodiments, the display lens 106 can include an arrangementof electrodes (e.g., an electrode array, a two-dimensional grid ofelectrodes) that are electrically connected to the electrical system132. The electrical system 132 can provide an electrical signal (e.g., avoltage signal or a current signal) to the electrodes in a portion ofthe display lens 106 that is configured to change the intensity ofambient light, change the spectral content of ambient light and/orchange the direction of ambient light incident on the display lens 106.The light emitting module 134, the thermal source 136, thesonic/ultrasonic transducer 138, and/or the electrical system 132 can beintegrated with the frame 80 as shown in FIG. 2B. Alternatively, in someembodiments one or all the light emitting module 134 the thermal source136, the sonic/ultrasonic transducer 138 and the electrical system 132can be positioned remotely from the display 70.

The perception of an image as being “three-dimensional” or “3-D” may beachieved by providing slightly different presentations of the image toeach eye of the viewer. FIG. 3 illustrates a conventional display systemfor simulating three-dimensional imagery for a user. Two distinct images190, 200—one for each eye 210, 220—are outputted to the user. The images190, 200 are spaced from the eyes 210, 220 by a distance 230 along anoptical or z-axis that is parallel to the line of sight of the viewer.The images 190, 200 are flat and the eyes 210, 220 may focus on theimages by assuming a single accommodated state. Such 3-D display systemsrely on the human visual system to combine the images 190, 200 toprovide a perception of depth and/or scale for the combined image.

It will be appreciated, however, that the human visual system is morecomplicated and providing a realistic perception of depth is morechallenging. For example, many viewers of conventional “3-D” displaysystems find such systems to be uncomfortable or may not perceive asense of depth at all. Without being limited by theory, it is believedthat viewers of an object may perceive the object as being“three-dimensional” due to a combination of vergence and accommodation.Vergence movements (i.e., rotation of the eyes so that the pupils movetoward or away from each other to converge the lines of sight of theeyes to fixate upon an object) of the two eyes relative to each otherare closely associated with focusing (or “accommodation”) of the lensesand pupils of the eyes. Under normal conditions, changing the focus ofthe lenses of the eyes, or accommodating the eyes, to change focus fromone object to another object at a different distance will automaticallycause a matching change in vergence to the same distance, under arelationship known as the “accommodation-vergence reflex,” as well aspupil dilation or constriction. Likewise, a change in vergence willtrigger a matching change in accommodation of lens shape and pupil size,under normal conditions. As noted herein, many stereoscopic or “3-D”display systems display a scene using slightly different presentations(and, so, slightly different images) to each eye such that athree-dimensional perspective is perceived by the human visual system.Such systems are uncomfortable for many viewers, however, since they,among other things, simply provide a different presentation of a scene,but with the eyes viewing all the image information at a singleaccommodated state, and work against the “accommodation-vergencereflex.” Display systems that provide a better match betweenaccommodation and vergence may form more realistic and comfortablesimulations of three-dimensional imagery contributing to increasedduration of wear and in turn compliance to diagnostic and therapyprotocols.

FIG. 4 illustrates aspects of an approach for simulatingthree-dimensional imagery using multiple depth planes. With reference toFIG. 4, objects at various distances from eyes 210, 220 on the z-axisare accommodated by the eyes 210, 220 so that those objects are infocus. The eyes 210, 220 assume particular accommodated states to bringinto focus objects at different distances along the z-axis.Consequently, a particular accommodated state may be said to beassociated with a particular one of depth planes 240, with has anassociated focal distance, such that objects or parts of objects in aparticular depth plane are in focus when the eye is in the accommodatedstate for that depth plane. In some embodiments, three-dimensionalimagery may be simulated by providing different presentations of animage for each of the eyes 210, 220, and also by providing differentpresentations of the image corresponding to each of the depth planes.While shown as being separate for clarity of illustration, it will beappreciated that the fields of view of the eyes 210, 220 may overlap,for example, as distance along the z-axis increases. In addition, whileshown as flat for ease of illustration, it will be appreciated that thecontours of a depth plane may be curved in physical space, such that allfeatures in a depth plane are in focus with the eye in a particularaccommodated state.

The distance between an object and the eye 210 or 220 may also changethe amount of divergence of light from that object, as viewed by thateye. FIGS. 5A-5C illustrate relationships between distance and thedivergence of light rays. The distance between the object and the eye210 is represented by, in order of decreasing distance, R1, R2, and R3.As shown in FIGS. 5A-5C, the light rays become more divergent asdistance to the object decreases. As distance increases, the light raysbecome more collimated. Stated another way, it may be said that thelight field produced by a point (the object or a part of the object) hasa spherical wavefront curvature, which is a function of how far away thepoint is from the eye of the user. The curvature increases withdecreasing distance between the object and the eye 210. Consequently, atdifferent depth planes, the degree of divergence of light rays is alsodifferent, with the degree of divergence increasing with decreasingdistance between depth planes and the viewer's eye 210. While only asingle eye 210 is illustrated for clarity of illustration in FIGS. 5A-5Cand other figures herein, it will be appreciated that the discussionsregarding eye 210 may be applied to both eyes 210 and 220 of a viewer.

Without being limited by theory, it is believed that the human eyetypically can interpret a finite number of depth planes to provide depthperception. Consequently, a highly believable simulation of perceiveddepth may be achieved by providing, to the eye, different presentationsof an image corresponding to each of these limited number of depthplanes. The different presentations may be separately focused by theviewer's eyes, thereby helping to provide the user with depth cues basedon the accommodation of the eye required to bring into focus differentimage features for the scene located on different depth plane and/orbased on observing different image features on different depth planesbeing out of focus.

FIG. 6 illustrates an example of a waveguide stack for outputting imageinformation to a user. A display system 250 includes a stack ofwaveguides, or stacked waveguide assembly, 260 that may be utilized toprovide three-dimensional perception to the eye/brain using a pluralityof waveguides 270, 280, 290, 300, 310. In some embodiments, the displaysystem 250 is the system 60 of FIG. 2A and/or FIG. 2B, with FIG. 6schematically showing some parts of that system 60 in greater detail.For example, the waveguide assembly 260 may be part of the display 70 ofFIG. 2A. As another example, the waveguide assembly 260 may be part ofthe display lens 106 of FIG. 2B. It will be appreciated that the displaysystem 250 may be considered a light field display in some embodiments.

With continued reference to FIG. 6, the waveguide assembly 260 may alsoinclude a plurality of features 320, 330, 340, 350 between thewaveguides. In some embodiments, the features 320, 330, 340, 350 may beone or more lenses. The waveguides 270, 280, 290, 300, 310 and/or theplurality of lenses 320, 330, 340, 350 may be configured to send imageinformation to the eye with various levels of wavefront curvature orlight ray divergence. Each waveguide level may be associated with aparticular depth plane and may be configured to output image informationcorresponding to that depth plane. Image injection devices 360, 370,380, 390, 400 may function as a source of light for the waveguides andmay be utilized to inject image information into the waveguides 270,280, 290, 300, 310, each of which may be configured, as describedherein, to distribute incoming light across each respective waveguide,for output toward the eye 210. Light exits an output surface 410, 420,430, 440, 450 of the image injection devices 360, 370, 380, 390, 400 andis injected into a corresponding input surface 460, 470, 480, 490, 500of the waveguides 270, 280, 290, 300, 310. In some embodiments, the eachof the input surfaces 460, 470, 480, 490, 500 may be an edge of acorresponding waveguide, or may be part of a major surface of thecorresponding waveguide (that is, one of the waveguide surfaces directlyfacing the world 510 or the viewer's eye 210). In some embodiments, asingle beam of light (e.g. a collimated beam) may be injected into eachwaveguide to output an entire field of cloned collimated beams that aredirected toward the eye 210 at particular angles (and amounts ofdivergence) corresponding to the depth plane associated with aparticular waveguide. In some embodiments, a single one of the imageinjection devices 360, 370, 380, 390, 400 may be associated with andinject light into a plurality (e.g., three) of the waveguides 270, 280,290, 300, 310.

In some embodiments, the image injection devices 360, 370, 380, 390, 400are discrete displays that each produce image information for injectioninto a corresponding waveguide 270, 280, 290, 300, 310, respectively. Insome other embodiments, the image injection devices 360, 370, 380, 390,400 are the output ends of a single multiplexed display which may, e.g.,pipe image information via one or more optical conduits (such as fiberoptic cables) to each of the image injection devices 360, 370, 380, 390,400. It will be appreciated that the image information provided by theimage injection devices 360, 370, 380, 390, 400 may include light ofdifferent wavelengths, or colors (e.g., different component colors, asdiscussed herein). In some embodiments, the image injection devices 360,370, 380, 390, 400 can be a part of the light projection systems 118 ofFIG. 2B.

In some embodiments, the light injected into the waveguides 270, 280,290, 300, 310 is provided by a light projector system 520, whichcomprises a light module 530, which may include a light emitter, such asa light emitting diode (LED). The light from the light module 530 may bedirected to and modified by a light modulator 540, e.g., a spatial lightmodulator, via a beam splitter 550. The light modulator 540 may beconfigured to change the perceived intensity of the light injected intothe waveguides 270, 280, 290, 300, 310. Examples of spatial lightmodulators include liquid crystal displays (LCD) including a liquidcrystal on silicon (LCOS) displays. In some embodiments, the lightprojector system 520 can be a part of the light projection systems 118of FIG. 2B.

In some embodiments, the display system 250 may be a scanning fiberdisplay comprising one or more scanning fibers configured to projectlight in various patterns (e.g., raster scan, spiral scan, Lissajouspatterns, etc.) into one or more waveguides 270, 280, 290, 300, 310 andultimately to the eye 210 of the viewer. In some embodiments, theillustrated image injection devices 360, 370, 380, 390, 400 mayschematically represent a single scanning fiber or a bundle of scanningfibers configured to inject light into one or a plurality of thewaveguides 270, 280, 290, 300, 310. In some other embodiments, theillustrated image injection devices 360, 370, 380, 390, 400 mayschematically represent a plurality of scanning fibers or a plurality ofbundles of scanning fibers, each of which are configured to inject lightinto an associated one of the waveguides 270, 280, 290, 300, 310. Itwill be appreciated that one or more optical fibers may be configured totransmit light from the light module 530 to the one or more waveguides270, 280, 290, 300, 310. It will be appreciated that one or moreintervening optical structures may be provided between the scanningfiber, or fibers, and the one or more waveguides 270, 280, 290, 300, 310to, e.g., redirect light exiting the scanning fiber into the one or morewaveguides 270, 280, 290, 300, 310.

A controller 560 controls the operation of one or more of the stackedwaveguide assembly 260, including operation of the image injectiondevices 360, 370, 380, 390, 400, the light source 530, and the lightmodulator 540. In some embodiments, the controller 560 is part of thelocal data processing module 140. The controller 560 includesprogramming (e.g., instructions in a non-transitory medium) thatregulates the timing and provision of image information to thewaveguides 270, 280, 290, 300, 310 according to, e.g., any of thevarious schemes disclosed herein. In some embodiments, the controllermay be a single integral device, or a distributed system connected bywired or wireless communication channels. The controller 560 may be partof the processing modules 140 or 150 (FIG. 2A) in some embodiments.

With continued reference to FIG. 6, the waveguides 270, 280, 290, 300,310 may be configured to propagate light within each respectivewaveguide by total internal reflection (TIR). The waveguides 270, 280,290, 300, 310 may each be planar or have another shape (e.g., curved),with major top and bottom surfaces and edges extending between thosemajor top and bottom surfaces. In the illustrated configuration, thewaveguides 270, 280, 290, 300, 310 may each include out-coupling opticalelements 570, 580, 590, 600, 610 that are configured to extract lightout of a waveguide by redirecting the light, propagating within eachrespective waveguide, out of the waveguide to output image informationto the eye 210. Extracted light may also be referred to as out-coupledlight and the out-coupling optical elements light may also be referredto light extracting optical elements. An extracted beam of light may beoutputted by the waveguide at locations at which the light propagatingin the waveguide strikes a light extracting optical element. Theout-coupling optical elements 570, 580, 590, 600, 610 may, for example,be gratings, including diffractive optical features, as discussedfurther herein. While illustrated disposed at the bottom major surfacesof the waveguides 270, 280, 290, 300, 310, for ease of description anddrawing clarity, in some embodiments, the out-coupling optical elements570, 580, 590, 600, 610 may be disposed at the top and/or bottom majorsurfaces, and/or may be disposed directly in the volume of thewaveguides 270, 280, 290, 300, 310, as discussed further herein. In someembodiments, the out-coupling optical elements 570, 580, 590, 600, 610may be formed in a layer of material that is attached to a transparentsubstrate to form the waveguides 270, 280, 290, 300, 310. In some otherembodiments, the waveguides 270, 280, 290, 300, 310 may be a monolithicpiece of material and the out-coupling optical elements 570, 580, 590,600, 610 may be formed on a surface and/or in the interior of that pieceof material.

With continued reference to FIG. 6, as discussed herein, each waveguide270, 280, 290, 300, 310 is configured to output light to form an imagecorresponding to a particular depth plane. For example, the waveguide270 nearest the eye may be configured to deliver collimated light (whichwas injected into such waveguide 270), to the eye 210. The collimatedlight may be representative of the optical infinity focal plane. Thenext waveguide up 280 may be configured to send out collimated lightwhich passes through the first lens 350 (e.g., a negative lens) beforeit can reach the eye 210; such first lens 350 may be configured tocreate a slight convex wavefront curvature so that the eye/braininterprets light coming from that next waveguide up 280 as coming from afirst focal plane closer inward toward the eye 210 from opticalinfinity. Similarly, the third up waveguide 290 passes its output lightthrough both the first 350 and second 340 lenses before reaching the eye210; the combined optical power of the first 350 and second 340 lensesmay be configured to create another incremental amount of wavefrontcurvature so that the eye/brain interprets light coming from the thirdwaveguide 290 as coming from a second focal plane that is even closerinward toward the person from optical infinity than was light from thenext waveguide up 280.

The other waveguide layers 300, 310 and lenses 330, 320 are similarlyconfigured, with the highest waveguide 310 in the stack sending itsoutput through all of the lenses between it and the eye for an aggregatefocal power representative of the closest focal plane to the person. Tocompensate for the stack of lenses 320, 330, 340, 350 whenviewing/interpreting light coming from the world 510 on the other sideof the stacked waveguide assembly 260, a compensating lens layer 620 maybe disposed at the top of the stack to compensate for the aggregatepower of the lens stack 320, 330, 340, 350 below. Such a configurationprovides as many perceived focal planes as there are availablewaveguide/lens pairings. Both the out-coupling optical elements of thewaveguides and the focusing aspects of the lenses may be static (i.e.,not dynamic or electro-active). In some alternative embodiments, eitheror both may be dynamic using electro-active features.

In some embodiments, two or more of the waveguides 270, 280, 290, 300,310 may have the same associated depth plane. For example, multiplewaveguides 270, 280, 290, 300, 310 may be configured to output imagesset to the same depth plane, or multiple subsets of the waveguides 270,280, 290, 300, 310 may be configured to output images set to the sameplurality of depth planes, with one set for each depth plane. This canprovide advantages for forming a tiled image to provide an expandedfield of view at those depth planes.

With continued reference to FIG. 6, the out-coupling optical elements570, 580, 590, 600, 610 may be configured to both redirect light out oftheir respective waveguides and to output this light with theappropriate amount of divergence or collimation for a particular depthplane associated with the waveguide. As a result, waveguides havingdifferent associated depth planes may have different configurations ofout-coupling optical elements 570, 580, 590, 600, 610, which outputlight with a different amount of divergence depending on the associateddepth plane. In some embodiments, the light extracting optical elements570, 580, 590, 600, 610 may be volumetric or surface features, which maybe configured to output light at specific angles. For example, the lightextracting optical elements 570, 580, 590, 600, 610 may be volumeholograms, surface holograms, and/or diffraction gratings. In someembodiments, the features 320, 330, 340, 350 may not be lenses; rather,they may simply be spacers (e.g., cladding layers and/or structures forforming air gaps).

In some embodiments, the out-coupling optical elements 570, 580, 590,600, 610 are diffractive features that form a diffraction pattern, or“diffractive optical element” (also referred to herein as a “DOE”).Preferably, the DOE's have a sufficiently low diffraction efficiency sothat only a portion of the light of the beam is deflected away towardthe eye 210 with each intersection of the DOE, while the rest continuesto move through a waveguide via TIR. The light carrying the imageinformation is thus divided into a number of related exit beams thatexit the waveguide at a multiplicity of locations and the result is afairly uniform pattern of exit emission toward the eye 210 for thisparticular collimated beam bouncing around within a waveguide.

In some embodiments, one or more DOEs may be switchable between “on”states in which they actively diffract, and “off” states in which theydo not significantly diffract. For instance, a switchable DOE maycomprise a layer of polymer dispersed liquid crystal, in whichmicrodroplets comprise a diffraction pattern in a host medium, and therefractive index of the microdroplets may be switched to substantiallymatch the refractive index of the host material (in which case thepattern does not appreciably diffract incident light) or themicrodroplet may be switched to an index that does not match that of thehost medium (in which case the pattern actively diffracts incidentlight).

In some embodiments, a camera assembly 630 (e.g., a digital camera,including visible light and infrared light cameras) may be provided tocapture images of the eye 210 and/or tissue around the eye 210 to, e.g.,detect user inputs and/or to monitor the physiological state of theuser. In various embodiments, the camera assembly 630 can be a part ofthe inward facing cameras 114 of FIG. 2B. As used herein, a camera maybe any image capture device. In some embodiments, the camera assembly630 may include an image capture device and a light source to projectlight (e.g., infrared light) to the eye, which may then be reflected bythe eye and detected by the image capture device. In some embodiments,the camera assembly 630 may be attached to the frame 80 (FIG. 2A) andmay be in electrical communication with the processing modules 140and/or 150, which may process image information from the camera assembly630 to make various determinations regarding, e.g., the physiologicalstate of the user, as discussed herein. It will be appreciated thatinformation regarding the physiological state of user may be used todetermine the behavioral or emotional state of the user. Examples ofsuch information include movements of the user and/or facial expressionsof the user. The behavioral or emotional state of the user may then betriangulated with collected environmental and/or virtual content data soas to determine relationships between the behavioral or emotional state,physiological state, and environmental or virtual content data. In someembodiments, one camera assembly 630 may be utilized for each eye, toseparately monitor each eye.

With reference now to FIG. 7, an example of exit beams outputted by awaveguide is shown. One waveguide is illustrated, but it will beappreciated that other waveguides in the waveguide assembly 260 (FIG. 6)may function similarly, where the waveguide assembly 260 includesmultiple waveguides. Light 640 is injected into the waveguide 270 at theinput surface 460 of the waveguide 270 and propagates within thewaveguide 270 by TIR. At points where the light 640 impinges on the DOE570, a portion of the light exits the waveguide as exit beams 650. Theexit beams 650 are illustrated as substantially parallel but, asdiscussed herein, they may also be redirected to propagate to the eye210 at an angle (e.g., forming divergent exit beams), depending on thedepth plane associated with the waveguide 270. It will be appreciatedthat substantially parallel exit beams may be indicative of a waveguideincluding out-coupling optical elements that out-couple light to formimages that appear to be set on a depth plane at a large distance (e.g.,optical infinity) from the eye 210. Other waveguides or other sets ofout-coupling optical elements may output an exit beam pattern that ismore divergent, which would require the eye 210 to accommodate to acloser distance to bring it into focus on the retina and would beinterpreted by the brain as light from a distance closer to the eye 210than optical infinity. In various embodiments, the exit beams 650 cancorrespond to the projection beam 124 of FIG. 2B.

In some embodiments, a full color image may be formed at each depthplane by overlaying images in each of the component colors, e.g., threeor more component colors. FIG. 8 illustrates an example of a stackedwaveguide assembly in which each depth plane includes images formedusing multiple different component colors. The illustrated embodimentshows depth planes 240 a-240 f, although more or fewer depths are alsocontemplated. Each depth plane may have three or more component colorimages associated with it, including: a first image of a first color, G;a second image of a second color, R; and a third image of a third color,B. Different depth planes are indicated in the figure by differentnumbers for diopters (dpt) following the letters G, R, and B. Just asexamples, the numbers following each of these letters indicate diopters(1/m), or inverse distance of the depth plane from a viewer, and eachbox in the figures represents an individual component color image. Insome embodiments, to account for differences in the eye's focusing oflight of different wavelengths, the exact placement of the depth planesfor different component colors may vary. For example, differentcomponent color images for a given depth plane may be placed on depthplanes corresponding to different distances from the user. Such anarrangement may increase visual acuity and user comfort and/or maydecrease chromatic aberrations.

In some embodiments, light of each component color may be outputted by asingle dedicated waveguide and, consequently, each depth plane may havemultiple waveguides associated with it. In such embodiments, each box inthe figures including the letters G, R, or B may be understood torepresent an individual waveguide, and three waveguides may be providedper depth plane where three component color images are provided perdepth plane. While the waveguides associated with each depth plane areshown adjacent to one another in this drawing for ease of description,it will be appreciated that, in a physical device, the waveguides mayall be arranged in a stack with one waveguide per level. In some otherembodiments, multiple component colors may be outputted by the samewaveguide, such that, e.g., only a single waveguide may be provided perdepth plane.

With continued reference to FIG. 8, in some embodiments, G is the colorgreen, R is the color red, and B is the color blue. In some otherembodiments, other colors associated with other wavelengths of light,including magenta and cyan, may be used in addition to or may replaceone or more of red, green, or blue. In some embodiments, features 320,330, 340, and 350 may be active or passive optical filters configured toblock or selectively light from the ambient environment to the viewer'seyes.

It will be appreciated that references to a given color of lightthroughout this disclosure will be understood to encompass light of oneor more wavelengths within a range of wavelengths of light that areperceived by a viewer as being of that given color. For example, redlight may include light of one or more wavelengths in the range of about620-780 nm, green light may include light of one or more wavelengths inthe range of about 492-577 nm, and blue light may include light of oneor more wavelengths in the range of about 435-493 nm.

In some embodiments, the light source 530 (FIG. 6) may be configured toemit light of one or more wavelengths outside the visual perceptionrange of the viewer, for example, infrared and/or ultravioletwavelengths. In addition, the in-coupling, out-coupling, and other lightredirecting structures of the waveguides of the display 250 may beconfigured to direct and emit this light out of the display towards theuser's eye 210, e.g., for imaging and/or user stimulation applications.

With reference now to FIG. 9A, in some embodiments, light impinging on awaveguide may need to be redirected to in-couple that light into thewaveguide. An in-coupling optical element may be used to redirect andin-couple the light into its corresponding waveguide. FIG. 9Aillustrates a cross-sectional side view of an example of a plurality orset 660 of stacked waveguides that each includes an in-coupling opticalelement. The waveguides may each be configured to output light of one ormore different wavelengths, or one or more different ranges ofwavelengths. It will be appreciated that the stack 660 may correspond tothe stack 260 (FIG. 6) and the illustrated waveguides of the stack 660may correspond to part of the plurality of waveguides 270, 280, 290,300, 310, except that light from one or more of the image injectiondevices 360, 370, 380, 390, 400 is injected into the waveguides from aposition that requires light to be redirected for in-coupling.

The illustrated set 660 of stacked waveguides includes waveguides 670,680, and 690. Each waveguide includes an associated in-coupling opticalelement (which may also be referred to as a light input area on thewaveguide), with, e.g., in-coupling optical element 700 disposed on amajor surface (e.g., an upper major surface) of waveguide 670,in-coupling optical element 710 disposed on a major surface (e.g., anupper major surface) of waveguide 680, and in-coupling optical element720 disposed on a major surface (e.g., an upper major surface) ofwaveguide 690. In some embodiments, one or more of the in-couplingoptical elements 700, 710, 720 may be disposed on the bottom majorsurface of the respective waveguide 670, 680, 690 (particularly wherethe one or more in-coupling optical elements are reflective, deflectingoptical elements). As illustrated, the in-coupling optical elements 700,710, 720 may be disposed on the upper major surface of their respectivewaveguide 670, 680, 690 (or the top of the next lower waveguide),particularly where those in-coupling optical elements are transmissive,deflecting optical elements. In some embodiments, the in-couplingoptical elements 700, 710, 720 may be disposed in the body of therespective waveguide 670, 680, 690. In some embodiments, as discussedherein, the in-coupling optical elements 700, 710, 720 are wavelengthselective, such that they selectively redirect one or more wavelengthsof light, while transmitting other wavelengths of light. Whileillustrated on one side or corner of their respective waveguide 670,680, 690, it will be appreciated that the in-coupling optical elements700, 710, 720 may be disposed in other areas of their respectivewaveguide 670, 680, 690 in some embodiments.

As illustrated, the in-coupling optical elements 700, 710, 720 may belaterally offset from one another. In some embodiments, each in-couplingoptical element may be offset such that it receives light without thatlight passing through another in-coupling optical element. For example,each in-coupling optical element 700, 710, 720 may be configured toreceive light from a different image injection device 360, 370, 380,390, and 400 as shown in FIG. 6, and may be separated (e.g., laterallyspaced apart) from other in-coupling optical elements 700, 710, 720 suchthat it substantially does not receive light from the other ones of thein-coupling optical elements 700, 710, 720.

Each waveguide also includes associated light distributing elements,with, e.g., light distributing elements 730 disposed on a major surface(e.g., a top major surface) of waveguide 670, light distributingelements 740 disposed on a major surface (e.g., a top major surface) ofwaveguide 680, and light distributing elements 750 disposed on a majorsurface (e.g., a top major surface) of waveguide 690. In some otherembodiments, the light distributing elements 730, 740, 750, may bedisposed on a bottom major surface of associated waveguides 670, 680,690, respectively. In some other embodiments, the light distributingelements 730, 740, 750, may be disposed on both top and bottom majorsurface of associated waveguides 670, 680, 690, respectively; or thelight distributing elements 730, 740, 750, may be disposed on differentones of the top and bottom major surfaces in different associatedwaveguides 670, 680, 690, respectively.

The waveguides 670, 680, 690 may be spaced apart and separated by, e.g.,gas, liquid, and/or solid layers of material. For example, asillustrated, layer 760 a may separate waveguides 670 and 680; and layer760 b may separate waveguides 680 and 690. In some embodiments, thelayers 760 a and 760 b are formed of low refractive index materials(that is, materials having a lower refractive index than the materialforming the immediately adjacent one of waveguides 670, 680, 690).Preferably, the refractive index of the material forming the layers 760a, 760 b is 0.05 or more, or 0.10 or less than the refractive index ofthe material forming the waveguides 670, 680, 690. Advantageously, thelower refractive index layers 760 a, 760 b may function as claddinglayers that facilitate TIR of light through the waveguides 670, 680, 690(e.g., TIR between the top and bottom major surfaces of each waveguide).In some embodiments, the layers 760 a, 760 b are formed of air. Whilenot illustrated, it will be appreciated that the top and bottom of theillustrated set 660 of waveguides may include immediately neighboringcladding layers.

Preferably, for ease of manufacturing and other considerations, thematerial forming the waveguides 670, 680, 690 are similar or the same,and the material forming the layers 760 a, 760 b are similar or thesame. In some embodiments, the material forming the waveguides 670, 680,690 may be different between one or more waveguides, and/or the materialforming the layers 760 a, 760 b may be different, while still holding tothe various refractive index relationships noted above.

With continued reference to FIG. 9A, light rays 770, 780, 790 areincident on the set 660 of waveguides. It will be appreciated that thelight rays 770, 780, 790 may be injected into the waveguides 670, 680,690 by one or more image injection devices 360, 370, 380, 390, 400 (FIG.6).

In some embodiments, the light rays 770, 780, 790 have differentproperties, e.g., different wavelengths or different ranges ofwavelengths, which may correspond to different colors. The in-couplingoptical elements 700, 710, 720 each deflect the incident light such thatthe light propagates through a respective one of the waveguides 670,680, 690 by TIR.

For example, in-coupling optical element 700 may be configured todeflect ray 770, which has a first wavelength or range of wavelengths.Similarly, the transmitted ray 780 impinges on and is deflected by thein-coupling optical element 710, which is configured to deflect light ofa second wavelength or range of wavelengths. Likewise, the ray 790 isdeflected by the in-coupling optical element 720, which is configured toselectively deflect light of third wavelength or range of wavelengths.

With continued reference to FIG. 9A, the deflected light rays 770, 780,790 are deflected so that they propagate through a correspondingwaveguide 670, 680, 690; that is, the in-coupling optical elements 700,710, 720 of each waveguide deflects light into that correspondingwaveguide 670, 680, 690 to in-couple light into that correspondingwaveguide. The light rays 770, 780, 790 are deflected at angles thatcause the light to propagate through the respective waveguide 670, 680,690 by TIR. The light rays 770, 780, 790 propagate through therespective waveguide 670, 680, 690 by TIR until impinging on thewaveguide's corresponding light distributing elements 730, 740, 750.

With reference now to FIG. 9B, a perspective view of an example of theplurality of stacked waveguides of FIG. 9A is illustrated. As notedabove, the in-coupled light rays 770, 780, 790, are deflected by thein-coupling optical elements 700, 710, 720, respectively, and thenpropagate by TIR within the waveguides 670, 680, 690, respectively. Thelight rays 770, 780, 790 then impinge on the light distributing elements730, 740, 750, respectively. The light distributing elements 730, 740,750 deflect the light rays 770, 780, 790 so that they propagate towardsthe out-coupling optical elements 800, 810, 820, respectively.

In some embodiments, the light distributing elements 730, 740, 750 areorthogonal pupil expanders (OPE's). In some embodiments, the OPE's bothdeflect or distribute light to the out-coupling optical elements 800,810, 820 and also increase the beam or spot size of this light as itpropagates to the out-coupling optical elements. In some embodiments,e.g., where the beam size is already of a desired size, the lightdistributing elements 730, 740, 750 may be omitted and the in-couplingoptical elements 700, 710, 720 may be configured to deflect lightdirectly to the out-coupling optical elements 800, 810, 820. Forexample, with reference to FIG. 9A, the light distributing elements 730,740, 750 may be replaced with out-coupling optical elements 800, 810,820, respectively. In some embodiments, the out-coupling opticalelements 800, 810, 820 are exit pupils (EP's) or exit pupil expanders(EPE's) that direct light in a viewer's eye 210 (FIG. 7). It will beappreciated that the OPE's may be configured to increase the dimensionsof the eye box in at least one axis and the EPE's may be to increase theeye box in an axis crossing, e.g., orthogonal to, the axis of the OPEs.

Accordingly, with reference to FIGS. 9A and 9B, in some embodiments, theset 660 of waveguides includes waveguides 670, 680, 690; in-couplingoptical elements 700, 710, 720; light distributing elements (e.g.,OPE's) 730, 740, 750; and out-coupling optical elements (e.g., EP's)800, 810, 820 for each component color. The waveguides 670, 680, 690 maybe stacked with an air gap/cladding layer between each one. Thein-coupling optical elements 700, 710, 720 redirect or deflect incidentlight (with different in-coupling optical elements receiving light ofdifferent wavelengths) into its waveguide. The light then propagates atan angle which will result in TIR within the respective waveguide 670,680, 690. In the example shown, light ray 770 (e.g., blue light) isdeflected by the first in-coupling optical element 700, and thencontinues to bounce down the waveguide, interacting with the lightdistributing element (e.g., OPE's) 730 and then the out-coupling opticalelement (e.g., EPs) 800, in a manner described earlier. The light rays780 and 790 (e.g., green and red light, respectively) will pass throughthe waveguide 670, with light ray 780 impinging on and being deflectedby in-coupling optical element 710. The light ray 780 then bounces downthe waveguide 680 via TIR, proceeding on to its light distributingelement (e.g., OPEs) 740 and then the out-coupling optical element(e.g., EP's) 810. Finally, light ray 790 (e.g., red light) passesthrough the waveguide 690 to impinge on the light in-coupling opticalelements 720 of the waveguide 690. The light in-coupling opticalelements 720 deflect the light ray 790 such that the light raypropagates to light distributing element (e.g., OPEs) 750 by TIR, andthen to the out-coupling optical element (e.g., EPs) 820 by TIR. Theout-coupling optical element 820 then finally out-couples the light ray790 to the viewer, who also receives the out-coupled light from theother waveguides 670, 680.

FIG. 9C illustrates a top-down plan view of an example of the pluralityof stacked waveguides of FIGS. 9A and 9B. As illustrated, the waveguides670, 680, 690, along with each waveguide's associated light distributingelement 730, 740, 750 and associated out-coupling optical element 800,810, 820, may be vertically aligned. However, as discussed herein, thein-coupling optical elements 700, 710, 720 are not vertically aligned;rather, the in-coupling optical elements are preferably non-overlapping(e.g., laterally spaced apart as seen in the top-down view). Asdiscussed further herein, this nonoverlapping spatial arrangementfacilitates the injection of light from different resources intodifferent waveguides on a one-to-one basis, thereby allowing a specificlight source to be uniquely coupled to a specific waveguide. In someembodiments, arrangements including nonoverlapping spatially-separatedin-coupling optical elements may be referred to as a shifted pupilsystem, and the in-coupling optical elements within these arrangementsmay correspond to sub pupils.

Display Systems with Regions of Variable Light Transmission

In embodiments of the display system 60 configured as augmented realityand/or virtual reality devices, contrast, brightness and/or clarity ofthe augmented reality content and/or virtual reality content that isdisplayed can be improved in a dim or dimmer environment. For example,contrast, brightness and/or clarity of augmented reality content and/orvirtual reality content can be reduced when embodiments of the displaysystem 60 configured as augmented reality and/or virtual reality devicesare viewed outside in bright sunlight, in brightly lit rooms, and/or inrainy/foggy environments with a lot of glare. Accordingly, it isadvantageous if the intensity of ambient light transmitted through aportion of the display 70 can be reduced when that portion of thedisplay 70 has glare and/or when the ambient light conditions over thatportion of the display 70 are bright to improve clarity of vision. Invarious embodiments, reducing the intensity of ambient light through aportion of the display 70 that is in an environment with bright ambientlight conditions can advantageously improve the user's visualexperience.

In some embodiments, the display system 60 can be configured to measurethe light intensity of bright ambient light sources, such as, forexample, but not limited to, desk lamps, overhead lights, street lights,car head lights, sun or combinations thereof and attenuate the amount oflight transmitted through one or more portions of the display 70 onwhich light from the bright ambient light sources is incident. Theamount of light from the bright ambient light sources that istransmitted through the one or more portions of the display 70 can bereduced by changing the transmissivity of the one or more portions ofthe display 70. For example, the one or more portions of the display 70may be darkened to reduce the amount of light from the bright ambientlight sources that is transmitted through the one or more portions. Insome implementations, the display 70 can comprise one or more opticalelements such as switchable light deflectors (e.g., optical zone plate,a diffractive optical element or a refractive optical element) that canbe switched to deflect some of the light from the bright ambient lightsources. The light may be deflected so as to reduce the amount of lightthat is incident on the eye or on the center of the retina (e.g., fovea)and in the center of the field of view of the viewer. As a result ofdeflecting light, the brightness the ambient light sources appears tothe viewer can be reduced and the contrast ratio of the virtual realitycontent can be increased. In various implementations, the transmissivityof light through the one or more portions of the display 70 need not bereduced to an amount that the bright ambient light sources are notvisible through the display. Instead, the transmissivity of lightthrough the one or more portions of the display 70 can be reduced to alevel that allows visibility of the virtual reality content withsufficient visual acuity and also allows visibility of the brightambient light sources.

Various embodiments of the display system 60 can comprise a forwardfacing camera/ambient light sensor that is configured to capture animage of a scene in the field of view (FOV) and determine the locationand intensity of various bright light sources in the scene. The forwardfacing camera can be associated with the display system 60. For example,the forward facing camera can be mounted on the display 70. Arelationship between the FOV of the camera and the FOV of the userthrough the display 70 can be determined. One or more portions of thedisplay 70 corresponding to the determined location of the bright lightsources in the scene that are configured to have reduced lighttransmissivity can be determined by determining the location of one ormore bright light sources in the FOV of the image captured by the cameraand identifying the locations of the display 70 corresponding to thosebright light sources.

A method of determining the location of the bright light sources in thescene and/or the intensity of the bright light sources in the scene canbe similar to the method of updating one or more settings of a contentcapture device using automatic exposure control (AEC) described in U.S.patent application Ser. No. 15/841,043, filed on Dec. 13, 2017, which isincorporated by reference herein in its entirety. Similar to the methodillustrated in FIG. 1A and described in paragraphs [0060]-[0065] of U.S.patent application Ser. No. 15/841,043, filed on Dec. 13, 2017, whichare incorporated by reference herein, the image captured by thecamera/ambient light sensor can be divided into a plurality of pixelgroups (e.g., 96 pixel groups, 120 pixel groups, 144 pixel groups,etc.). An average luma value can be computed for each pixel group asdescribed in paragraph [0065] of U.S. patent application Ser. No.15/841,043, filed on Dec. 13, 2017, which is incorporated by referenceherein. In some examples, an average luma pixel group value may becomputed by accumulating luma values for each pixel of a pixel group. Insuch examples, luma values may represent the brightness of an image(e.g., an achromatic portion of an image or a grey scale image).Accordingly, a luma value may be a representation of an image without acolor component. As another example, in a YUV colorspace, a luma valuemay be the Y. In some examples, a luma value is a weighted sum ofgamma-compressed RGB components of an image. In such examples, the lumavalue may be referred to as gamma-corrected luma. In some examples,accumulation may be performed by software or hardware by adding up lumavalues for each pixel of the pixel group. In some implementations, oncethe luma values for a pixel group are accumulated, the total number maybe divided by the number of pixels in the pixel group to compute anaverage luma pixel group value for the pixel group. This process may berepeated for each pixel group in the image.

If the image captured by the camera is a grayscale image, then the pixelvalue associated with the plurality of pixel groups of the grayscaleimage correspond to the average luma value. In some implementations,color images captured by the ambient light sensor can be converted toYUV image format and the luma value corresponding to the Y component ofthe YUV image can be determined.

In some implementations, one or more bright spots on the display 70 canbe identified to correspond to one or more saturation regions of theimage captured by the ambient light sensor. For example, one or morebright spots on the display 70 that corresponds to the position of thebright light sources in the scene can be determined based on a maximumallowable luma value difference between adjacent pixels, or adjacentgroups of pixels. The maximum allowable luma value difference betweenadjacent pixels can be calculated in different ways. For example, in onemethod pixels that have relative pixel values within a certain thresholdof each other can be grouped together. Another method of grouping therelative pixel values relies on adaptive k-means clustering algorithmwhich outputs a set of clusters with luma values above a certainthreshold level. In some implementations, saturation region cancorrespond to the portion of the image having luma value above athreshold value. The threshold value can, for example, be 220 for an8-bit image ranging from 0 for black to 255 for white. The portions ofthe display 70 that correspond to the portions of the image having lumavalues above a certain threshold can be selectively occluded to reducetransmissivity of the light from the bright light sources. Otherapproaches may be employed.

In some embodiments, the display system 60 can comprise an electronicprocessor (e.g., local processing & data module 140 and/or remoteprocessing module 150) that is configured to reduce the amount of lighttransmitted through the portions of the display 70 that receive lightfrom the locations of the ambient environment that have higher lightintensity than an average light intensity of the ambient environment. Inthis manner, the intensity of light transmitted through display 70 canbe reduced in portions of the display 70 that receive the most ambientlight. Additionally, the electronic processor can be configured todetermine the portions of the display 70 where the virtual realitycontent is displayed and reduce the amount of ambient light transmittedthrough those portions to increase the relative brightness of thevirtual reality content.

To facilitate selectively reducing the transmissivity of light throughone or more portions of the display 70, the display 70 can be configuredas a pixelated display. For example, the surface of the display 70 cancomprise a plurality of electronically addressable pixels that can beconfigured to vary the amount of light transmitted therethrough. In someimplementations, the plurality of electronically addressable pixels cancomprise a plurality of spatial light modulators. In someimplementations, the display 70 can comprise an occlusion mask a theplurality of electronically addressable pixels. The occlusion mask cancomprise a plurality of mask elements, each mask element beingassociated with one or more of the plurality of addressable pixels. Theplurality of mask elements can have different values associated with thedifferent values of transmissivity through the plurality ofelectronically addressable pixels. The electronic processor (e.g., localprocessing & data module 140 and/or remote processing module 150) can beconfigured to selectively reduce the amount of light transmitted throughone or more of the plurality of pixels to reduce brightness of ambientlight sources and/or to improve contrast ratio of the virtual realitycontent.

As discussed above, the display 70 can include a display lens 106. Invarious embodiments, the display lens 106 can be a unitary lenspositioned in front of both eyes of the user 90. The unitary lens canhave ocular regions positioned in front of each eye through which theuser can view the surrounding environment. In some embodiments, thedisplay lens 106 can comprise two lens elements, each lens elementpositioned in front of each eye of the user 90. Each lens element canhave an ocular region through which the user can view the surrounding.

Various embodiments described herein are configured to reduce intensityof light transmitted through one or more portions of the display lens106, such as, for example by absorbing some of the ambient lightincident on the portion of the display lens 106 and/or byscattering/refracting/diffracting some of the ambient light incident onthe portion(s) of the display lens 106 away from the pupil of the eye.Additionally, in embodiments of the display lens 106 comprising two lenselements positioned in front of each eye respectively, the intensity ofambient light transmitted through only one of the lens elements (or aportion or portions thereof) may be reduced. As another example, theintensity of ambient light transmitted through a portion of one or boththe ocular regions of the display lens 106 is reduced while theintensity of ambient light transmitted through the remainder of thedisplay lens 106 is not reduced (or is reduced but by a lesser amount).As yet another example, the intensity of ambient light transmittedthrough a first portion of the display lens 106 is reduced while theintensity of ambient light transmitted through a second portion of thedisplay lens is not reduced. In contrast to sunglasses that darkenuniformly in bright sunlight and lighten uniformly indoors, variousembodiments of the display lens 106 are configured to darken or lightennon-uniformly. For example, the display lens 106 may darken partially,e.g., only part of the lens 106 may darken. As another example, thedisplay lens 106 may darken by different amounts in different parts ofthe lens. Additionally, in various embodiments of the system 60, partialdarkening of portions of the display lens 106 may be achieved inresponse to a stimulus provided by the display system (e.g., opticalstimulus provided the light emitting module 134, electrical stimulusprovided by the electrical system 132, thermal energy provided by thethermal source 136 and/or sonic/ultrasonic energy provided by thesonic/ultrasonic transducer 138) based on information obtained by one ormore components that sense the user's environment such as, for example,light sensor 128, the sensor assembly 126, the outward facing cameras112 and/or inward facing cameras 114 in conjunction with data from anassociated cloud computing resource. In various embodiments of thedisplay system 60, darkening or lightening of the display lens 106 neednot occur automatically in response to ambient light conditions but inresponse to a stimulus provided by the display system (e.g., opticalstimulus provided by the light emitting module 134, electrical stimulusprovided by the electrical system 132, thermal energy provided by thethermal source 136 and/or sonic/ultrasonic energy provided by thesonic/ultrasonic transducer 138) based on environmental informationobtained by one or more cameras/sensors of the system 60 with/withoutdata from an associated cloud computing resource. In variousembodiments, at least one portion of the display lens 106 can beconfigured to transmit between about 1%-100% of incident ambient light.For example, the at least one portion of the display lens 106 can beconfigured to transmit between about 5%-90% of incident ambient light,between about 10%-80% of incident ambient light, between about 15%-75%of incident ambient light, between about 20%-70% of incident ambientlight, between about 25%-60% of incident ambient light, between about30%-50% of incident ambient light, or any value in these ranges and/orsub-ranges.

The display lens 106 can comprise at least one variable optical material(e.g., organic molecules, proteins, photochromic materials,electrochromic materials, silver compounds such as, for example, silverhalide or silver chloride molecules, aerosols, hydrocolloids, etc.) thatcan be activated using thermal, sonic/ultrasonic, optical and/orelectrical stimulus to vary at least one of: the intensity of ambientlight transmitted through the display lens 106, spectral content ofambient light transmitted through the display lens 106, or the opticalpath of the ambient light transmitted through the display lens 106(e.g., by diffraction, by scattering, by refraction or by changing therefractive index of the variable optical element). The variable opticalmaterial may comprise a layer of molecules or a plurality of layers ofmolecules. In various embodiments, the at least one variable opticalmaterial can comprise protein based electroactive materials that respondto an electrical stimulus (e.g., a voltage signal and/or a currentsignal) provided by the display system 60 to vary at least one of: theintensity of ambient light transmitted through the display lens 106,spectral content of ambient light transmitted through the display lens106, or the optical path of the ambient light transmitted through thedisplay lens 106. For example, in response to an electrical stimulusprovided by the display system 60, the protein based electroactivematerials can move, expand, contract, twist, rotate, adhere together ormove away from each other to vary at least one of: the intensity ofambient light transmitted through the display lens 106, spectral contentof ambient light transmitted through the display lens 106, or theoptical path of the ambient light transmitted through the display lens106. In some embodiments, the at least one variable optical material cancomprise organic materials (e.g., oxazines and/or naphthopyrans) thatvary at least one of: the intensity of ambient light transmitted throughthe display lens 106, spectral content of ambient light transmittedthrough the display lens 106, or the optical path of the ambient lighttransmitted through the display lens 106 in response to an opticalstimulus provided by the display system 60. The molecules of the organicmaterials can be configured to change their size and/or shape whenirradiated with light of certain frequencies or wavelengths (e.g., UVlight). For example, the organic materials can be configured to expandand absorb more light (therefore reducing intensity of light transmittedto the user) when irradiated with light of certain frequencies. Asanother example, the molecules of the organic materials can beconfigured to move, shrink, twist, rotate, clump together or move awayfrom each other to vary the intensity of light transmitted through thedisplay lens 106 in response to an optical stimulus. The molecules ofthe organic materials can vary the intensity of light transmittedthrough the display lens 106 by absorbing a portion of the lighttransmitted through the display lens 106, by changing the color of thedisplay lens 106 and/or by diffracting/refracting/scattering portion ofthe light transmitted away from the display lens 106. As discussedabove, the variable optical material may comprise a layer of moleculesor a plurality of layers of molecules.

In various embodiments, the at least one variable optical material cancomprise one or more molecules that are bound with certain chemicalsthat can be configured to vary the transmissivity of light in responseto a stimulus provided by the system 60. The chemicals bound to the oneor more molecules can be configured to vary intensity of incomingambient light, direction of incoming ambient light and/or spectralcontent of incoming ambient light when irradiated by specificwavelengths of light (e.g., UV, infrared and/or one or more wavelengthsin the visible spectrum).

Because the at least one variable optical material (e.g., photoreactiveand/or electroactive materials) are configured to vary at least one of:the intensity of ambient light transmitted through the display lens 106,spectral content of ambient light transmitted through the display lens106, or the optical path of the ambient light transmitted through thedisplay lens 106 in response to stimulus provided by the display system60, the location of the desired portion of the display lens 106 throughwhich the intensity of incoming ambient light, direction of incomingambient light and/or spectral content of incoming ambient light ischanged (e.g., by absorption in the desired portion, by changing colorof the desired portion and/or by diffraction/refraction/scattering ofthe ambient light away from the desired portion), the duration of timethat the desired portion of the display lens 106 is configured to changeintensity of incoming ambient light, direction of incoming ambient lightand/or spectral content of incoming ambient light and the speed at whichthe desired portion of the display lens 106 is darkened or lightened canbe controlled (e.g., precisely controlled).

Additionally, the distribution of the at least one variable opticalmaterial across the surface of the display lens 106 can be tailored tomeet certain requirements/functions. In some embodiments, the at leastone variable optical material can be distributed uniformly across thesurface of the display lens 106. In some other embodiments, the at leastone variable optical material can be distributed unevenly across thesurface of the display lens 106 such that portions of the display lens106 can have higher density of the at least one variable opticalmaterial as compared to other portions of the display lens 106. In someembodiments, the density of the at least one variable optical materialin portions of the ocular regions of the display lens 106 may be greaterthan in portions of the non-ocular regions (e.g., regions of the displaylens corresponding to the temples, nose bridge, eye orbitals and othernon-ocular portions of the user's face) which the user cannot seethrough. In some embodiments, certain regions of the display lens 106(e.g., the non-ocular regions) can be devoid of the at least onevariable optical material since it may not be necessary to vary at leastone of: the intensity of the ambient light, spectral content of theambient light, or the optical path of the ambient light in thoseregions.

Various embodiments of the display lens 106 can comprise a plurality oflayers, each layer including variable optical materials that vary atleast one of: the intensity of ambient light, spectral content ofambient light, or the optical path of the ambient light in response to astimulus provided by the display system 60. The materials of theplurality of layers may be configured to act on different wavelengths ofthe incoming ambient light. For example, the materials of the pluralityof layers may attenuate different wavelengths of the incoming ambientlight by different amounts. As another example, the materials of theplurality of layers may absorb different wavelengths of the incomingambient light by different amounts. As yet another example, thematerials of the plurality of layers may diffract/scatter/refractdifferent wavelengths of the incoming ambient light by differentamounts.

Accordingly, some embodiments of the display lens 106 can include afirst layer comprising a first variable optical material that isconfigured to attenuate (e.g., by absorption, diffraction, refraction,reflection or scattering) red light in response to a stimulus providedby the display system 60, a second layer comprising a second variableoptical material that is configured to attenuate (e.g., by absorption,diffraction, refraction, reflection or scattering) green light inresponse to a stimulus provided by the display system 60, a third layercomprising a third variable optical material that is configured toattenuate (e.g., by absorption, diffraction, refraction, reflection orscattering) blue light in response to a stimulus provided by the displaysystem 60, a fourth layer comprising a fourth variable optical materialthat is configured to attenuate (e.g., by absorption, diffraction,refraction, reflection or scattering) ultraviolet light in response to astimulus provided by the display system 60 and/or a fifth layercomprising a fifth variable optical material that is configured toattenuate (e.g., by absorption, diffraction, refraction, reflection orscattering) infrared light in response to a stimulus provided by thedisplay system 60. A subset of theses layers can alternatively beincluded in the display lens or display system. For example first,second, and third layers for attenuating, red, green, and blue lightrespectively. In such embodiments, thermal, sonic/ultrasonic, optical orelectrical stimulus can be provided to one or more of the plurality oflayers to attenuate (e.g., by absorption, diffraction, refraction,reflection or scattering) specific wavelengths of light based onenvironmental information obtained by one or more cameras/sensors of thesystem 60 with/without data from an associated cloud computing resource.

In various embodiments, groups of variable optical materials having thesame chemical/physical property can be individually activated to performa variety of functions without activating other groups of variableoptical materials having different chemical/physical property. Invarious embodiments, the variable optical materials that change at leastone of: intensity of ambient light, spectral content of ambient light oroptical path of ambient light incident on the display lens 106 can onlybe provided in certain portions of the display lens 106 (e.g., theocular portions of the display lens 106, a part of the ocular portionsof the display lens 106, only one of the ocular portions of the displaylens 106, etc.). In some such embodiments, the portions of the displaylens 106 comprising the variable optical materials may automaticallydarken/lighten in the presence/absence of sunlight without requiring anyadditional stimulus from the display system 60.

In various embodiments, the variable optical materials can be integratedwith the display lens 106. However, in some other embodiments, thevariable optical materials may be included in an add-on device that canbe attached or detached to the display lens 106. The embodiments ofdisplay lenses integrated with the variable optical materials and/oradd-on devices including the variable optical materials can beconfigured to be activated by a small amount of activation energy (e.g.,thermal, sonic/ultrasonic, optical and/or electrical energy). In somecases, after activation, the physical and/or chemical changes of themolecules of variable optical materials that changes at least one of:intensity of ambient light, spectral content of ambient light or opticalpath of ambient light may occur without requiring any additional amountof energy. The physical and/or chemical changes of the variable opticalmaterials may be maintained until the variable optical materials aredeactivated by providing deactivation energy (e.g., thermal,sonic/ultrasonic, optical and/or electrical energy).

As discussed above, in some implementations the display 70 can beconfigured as a pixelated display. For example, the surface of thedisplay 70 can comprise a plurality of electronically addressable pixelsthat can vary the amount of light transmitted therethrough in responseto an electrical or an optical stimulus. In some implementations, theplurality of electronically addressable pixels can comprise a pluralityof spatial light modulators. In some implementations, the display 70 cancomprise an occlusion mask comprising a plurality of mask elementsassociated with the plurality of electronically addressable pixels. Theelectronic processor (e.g., local processing & data module 140 and/orremote processing module 150) can be configured to provide an electricalor an optical signal to selectively reduce the amount of lighttransmitted through one or more of the plurality of pixels to reducebrightness of ambient light sources and/or to improve contrast ratio ofthe virtual reality content.

The following examples illustrate the advantages and the variousoperational characteristics of an embodiment of the display system 60that is configured to alter at least one of: intensity of ambient light,spectral content of ambient light and/or direction of ambient lightincident on the display 70 as described above. Consider an embodiment ofthe display system 60 comprising variable optical materials (eitherintegrated with the display lens 106 of the display system 60 orincluded in an add-on device) that is worn by the user 90. As the usermoves from a low ambient light condition (e.g., indoors) to a brightenvironment (e.g., outdoors), the sensors assemblies (e.g., lightsensors, outwards facing cameras, inward facing cameras, etc.) of thedisplay system 60 will detect the change in the ambient light condition.The sensor assemblies may be configured to detect change in ambientlight condition by detecting changes in the intensity of ambient lightas well as by detecting changes in the environment usinglocation-specific information (e.g., information obtained by a GPS, acompass and/or information obtained from an associated cloud computingresource), information regarding the surrounding environment obtainedusing object recognition algorithms to determine trees/park, buildings,rooms, etc., temperature sensors, etc. In addition to determining achange in the intensity of ambient light condition, the sensorassemblies may be configured to determine the spectral characteristicsof the incident light as well. The sensor assemblies may be configuredto determine the intensity/spectral characteristic of ambient light thatis incident on different portions of the display lens 106. The sensorassemblies may include sensors having filters and/or specific spectralresponses to determine the spectral characteristics of ambient orincident light. Accordingly, in certain embodiments, the sensorassemblies may be configured to locate and identify positions of variousambient light sources in the real world visible to the user through thedisplay 70 as well as identify portions of the display 70 and/or thedisplay lens 106 that are aligned with the ambient light sources for aparticular position of the user's head. Once the various portions of thedisplay lens 106 that coincide with the ambient light sources in thereal world is known, the system 60 can provide optical, electrical,thermal and/or sonic/ultrasonic stimulus to different portions of thedisplay lens 106 to cause a portion of the incident ambient light to beabsorbed, deflected, refracted, scattered and/or reflected such that theamount of ambient light transmitted through that portions of the displaylens 106 that coincide with the ambient light sources in the real worldis reduced or otherwise altered. In this manner, the amount of ambientlight transmitted through the display lens 106 can be varied across thesurface of the display lens 106 depending on the environmentalconditions. For example, consider that the user 90 is outside in themorning or evening hours when sunlight is incident on the display lensfrom one side of the user such that the amount of ambient light incidenton the surface of the display lens 106 is not uniform. In suchembodiments, the system 60 can be configured to transmit a greateramount of light through one portion of the display lens 106 than theamount of light transmitted through another portion of the display lens106. In various embodiments, the amount of light transmitted through oneportion of the display lens 106 can be about 1%-100% (e.g., 2%-95%,5%-90%, 7%-80%, 10%-75%, 15%-50%, 20%-60%, 30%-85%, etc.) greater thanthe amount of light transmitted through another portion of the displaylens 106.

In various embodiments, the information obtained from the varioussensors and/or camera assemblies can be sent to the local processing &data module 140 and/or the remote processing module 150 for processing.The local processing & data module 140 and/or the remote processingmodule 150 can determine one or more locations of the display lens 106that are aligned with different ambient light sources by processing theinformation obtained from the various sensors and/or camera assemblies.In some embodiments, the local processing & data module 140 and/or theremote processing module 150 can store the position of various objectsin the real world with respect to the display device and/or the user'shead/eyes in a database. The database can be updated or provideinformation in real time or in near real time as the objects in thesurrounding real world appear to move with respect to the display deviceand/or the user's head/eyes as the user moves his/her head. The databasecan be updated or provide information in real time or in near real timeregarding position with respect to the display device and/or the user'shead/eyes of new objects from the surrounding real world that come intothe user's field of view as the user moves his/her head. In variousembodiments, the local processing & data module 140 and/or the remoteprocessing module 150 can be configured to determine theintensity/spectral characteristics of the ambient light sources thatappear to be aligned with different portions of the display lens 106when viewed through the display lens 106. The local processing & datamodule 140 and/or the remote processing module 150 can be configured toreduce the amount of ambient light transmitted through the portions ofthe display lens 106 when viewed through the display lens 106 thatappear to be aligned with the ambient light sources. The localprocessing & data module 140 and/or the remote processing module 150 cansend signals that can trigger the light emitting module 134, theelectrical system 132, the thermal source 136 and/or thesonic/ultrasonic transducers 138 to provide the appropriate stimulus toactivate the variable optical element in the different portions of thedisplay lens 106 to attenuate ambient light in those portions by theappropriate amount. As discussed above, the light through differentportions of the display lens 106 can be attenuated by same or differentamounts depending on the intensity/spectral characteristics of the lightfrom the ambient light sources that appear to be aligned with thoseportions. This can be advantageous when light is incident on the user'seyes from one side, such as, for example from a desk lamp positioned onone side of the user, sunlight in the morning or evening hours, orobjects in the real world seen through different portions of the displaylens 106 that produce different amounts of glare.

In various embodiments, the system 60 can be configured to obtaininformation about the environment continuously or substantiallycontinuously. For example, the system 60 can be configured to obtaininformation about the environment from the various cameras/sensorassemblies at 1-30 microsecond intervals, at 100-500 microsecondintervals, 400 microseconds-1 millisecond intervals, at 1-30 millisecondintervals, at 20-100 millisecond intervals, at 50-500 millisecondintervals, at 400 millisecond-1 second intervals, at 1-5 secondintervals, or at any values in these ranges or sub-ranges or anycombinations thereof. The local processing & data module 140 and/or theremote processing module 150 can be configured to process theinformation obtained from the various cameras/sensor assemblies of thesystem 60 and send signals that can trigger the light emitting module134, the electrical system 132, the thermal source 136 and/or thesonic/ultrasonic transducers 138 to provide the required stimulus toactivate the variable optical element in the different portions of thedisplay lens 106 in real-time or near real-time, for example, such thatthe user experience is maintained as the environmental conditionschange.

For example, in various embodiments, the light sensors 128 can beconfigured to sense intensity and/or spectral characteristics of ambientlight incident on the display lens. Additionally, the outward facingcameras, the inward facing cameras and other sensor assemblies can beconfigured to obtain information about the surrounding world viewable tothe user through the display lens 106 that can help in identifyingdifferent sources of ambient light and/or glare producing objects in thereal world as well as their position with respect to the display 70,and/or the display lens 106 and/or the user's eye. In variousembodiments, the display system 60 can also be configured to identifythe nature of the ambient light source that appears to be aligned withdifferent portions of the display lens 106 (e.g., sunlight, fluorescentlight, incandescent light, LED light, candle). Once the system 60 hasidentified the position of the various ambient light sources withrespect to the display 70 and/or display lens 106 and/or the user's eye,it can determine portions of the display 70 and/or display lens 106whose light transmission characteristics should be changed in order tomaintain/improve user's visual experience. The system 60 can provide astimulus to the determined portions of the display lens 106 to attenuatelight transmitted through those portions in real time or near real timeand/or to change the direction or spectral characteristics of lighttransmitted through those portions in order, for example, tomaintain/improve user's visual experience. In this manner, the user'svisual experience need not be substantially compromised as a result ofglare or intensity changes across the surface of the display lens 106.

In various embodiments, the system 60 may be configured to store maps oflocations frequently visited by the user in a data repository accessibleby the local processing & data module 140 and/or the remote processingmodule 150. The stored map for one or more locations frequently visitedby the user can include positions of ambient light sources (e.g., streetlights, porch lights, traffic lights, etc.). Information about theintensity and/or spectral content of light from the ambient lightsources at one or more locations frequently visited can also be storedin the data repository. Information about how the light transmissioncharacteristics of various portions of the display lens 106 should bechanged at various times of the day, night and/or year may bepredetermined for one or more locations frequently visited by the userand stored in the data repository as well. For example, for a locationfrequently visited by the user, information about how the lighttransmission capability of different portions of the display lens 106that appear to be aligned with different ambient light sources at thatlocation should be changed during daytime can be stored in the datarepository. As another example, for a location frequently visited by theuser, information about how the light transmission capability ofdifferent portions of the display lens 106 that appear to be alignedwith different ambient light sources at that location should be changedduring nighttime (or any other time) can be stored in the datarepository. As yet another example, for a location frequently visited bythe user, information about how the light transmission capability ofdifferent portions of the display lens 106 that appear to be alignedwith different ambient light sources at that location should be changedduring daytime in summer can be stored in the data repository. Asanother example, for a location frequently visited by the user,information about how the light transmission capability of differentportions of the display lens 106 that appear to be aligned withdifferent ambient light sources at that location should be changedduring daytime in winter can be stored in the data repository. Thelocations and characteristics (e.g., size, shape, brightness, coloretc.) of the different light source at the different locations can alsobe recorded and stored for later access and use.

The local processing & data module 140 and/or the remote processingmodule 150 may be configured to identify the location from the sensorinformation; access the information from the data repository on thelocation and other characteristics of the light sources (e.g., size,shape, brightness, color etc.) as well as potentially how the lighttransmission capability of different portions of the display lens 106that appear to be aligned with different ambient light sources at thatlocation should be changed for that particular time of day and year.This information can be used to direct the stimulus providing sources toactivate the variable optical materials in various portions of thedisplay lens to change the intensity, spectral content and/or directionof ambient light in accordance with the predetermined information.

This can advantageously save processing time. For example, information(e.g., location, intensity, spectral content, etc.) about variousambient light sources (e.g., lamps, windows, over head lights, etc.) ina user's home or office can be stored in the data repository.Information regarding the location of the sun, the direction of sunlightat various time of the day can also be stored in the data repository.When the system 60 detects from the information obtained by the sensorsthat the user is in the office or home, the local processing & datamodule 140 and/or the remote processing module 150 can send appropriatesignals to the various stimulus providing sources to darken and/orlighten various portions of the display lens 70 based on the storedinformation (e.g., location, intensity, spectral content, etc.) aboutvarious ambient light sources in the user's home or office.

FIG. 10 illustrates a scene 1000 viewed by a user during nighttimethrough a display lens 1006 of an embodiment of a display system. Thedisplay system can have features similar to the display system 60discussed above. For example, the display system can include one or moresensors configured to obtain information of the scenes, the informationincluding position of the various ambient light sources with respect tothe display lens 1006, the brightness of the various ambient lightsources and/or the type of the various ambient light sources (e.g.,fluorescent light, LED light, incandescent light, etc.). The displaysystem can also include electronic processing systems configured toprocess the information obtained by the one or more sensors. Processingthe information obtained by the one or more sensors can includeidentifying the portions of the display lens 1006 that appear to bealigned with (or coincide with) the various ambient light sources in thescene 1000 viewed by the user and to determine the light transmissioncharacteristic of one or more portions of the display lens 1006 toimprove/maintain the user's visual experience. The display lens 1006comprises one or more variable optical materials that are configured tochange the intensity of incident ambient light, spectral content ofincident ambient light and/or direction of incident ambient light inresponse to an optical, electrical, thermal and/or sonic/ultrasonicstimulus provided by the display system. The display lens 1006 can havefeatures similar to the display lens 106. The scene 1000 includes afront porch of a house 1003 and several sources of ambient light 1005 a,1005 b, 1005 c, 1005 d, 1005 e and 1005 f along a sidewalk. The sourcesof ambient light 1005 a-1005 f can include porch lights, street lights,indoor lights, outdoor lights, path lights, landscape lighting, etc. Inembodiments of display lenses without one or more variable opticalmaterials, the sources of ambient light 1005 a-1005 f can produce glareand/or degrade the clarity of vision when viewed through the portions ofthe display lenses that appear to be aligned with the sources of ambientlight 1005 a-1005 f. In contrast, the display lens 1006 is configured tochange the intensity of ambient light, spectral content of the ambientlight and/or direction of the ambient light incident on the display lens1006 through the portions of the display lens 1006 that appear to bealigned with the sources of ambient light 1005 a-1005 f to reduceinterference with the user experience due to glare resulting from theambient light 1005 a-1005 f.

As discussed above, the sensors associated with the display system cancontinuously or intermittently obtain information of the scene 1000. Theinformation can include position of the ambient light sources 1005a-1005 f with respect to the display lens 1006, the direction, intensityand spectral content of ambient light from the ambient light sources1005 a-1005 f. The electronic processing systems can process theinformation obtained by the one or more sensors, determine how thedistribution of ambient light across the surface of the display lens1006 should be changed. For example, in some embodiments, the electronicprocessing systems can determine that an area of the display lens 1006(e.g., 1010 a, 1010 b, 1010 c, 1010 d, 1010 e and 1010 f) including theportion of the scene including the ambient light sources 1005 a-1005 fshould be darkened to reduce the intensity of ambient light transmittedthrough those portions. As another example, in some embodiments, theelectronic processing systems can determine that the incident ambientlight in an area of the display lens 1006 (e.g., 1010 a, 1010 b, 1010 c,1010 d, 1010 e and 1010 f) including the portion of the scene includingthe ambient light sources 1005 a-1005 f should be diffused to reduceglare. As another example, in some embodiments, the electronicprocessing systems can determine that the incident ambient light in anarea of the display lens 1006 (e.g., 1010 a, 1010 b, 1010 c, 1010 d,1010 e and 1010 f) including the ambient light sources 1005 a-1005 fshould be redirected to reduce glare.

Based on the determination, the electrical processing system can sendsignals to activate the optical, thermal, sonic/ultrasonic and/orelectrical source associated with the display system and provide adesired optical, thermal, sonic/ultrasonic and/or electrical stimulus tothe area of the display lens 1006 (e.g., 1010 a, 1010 b, 1010 c, 1010 d,1010 e and 1010 f) including the ambient light sources 1005 a-1005 fthat causes a physical and/or chemical change to the variable opticalmaterials in that area of the display lens which in turn can changeintensity of incident ambient light, spectral content of incidentambient light and/or direction of incident ambient light.

In various embodiments, the system 60 may be configured to track themovement of the user's eyes and/or head in real time or in near realtime and determine the relative position between real world objects(e.g., trees, sun, ambient light sources, etc.) and the user's eyes inreal time or in near real time. In such embodiments, the system 60 maybe configured to dynamically change the ambient light transmissioncharacteristics through different portions of the display lens as theuser's head and/or eyes move, e.g., to maintain/improve the user'svisual experience. For example, consider the user's head is in a firstposition and the ambient light source appears to be aligned with aportion of the display lens 106 to the left of the left eye pupil of theuser. If the user remains in the first head position, the portion of thedisplay lens 106 that is to the left of the left eye pupil may bedarkened or otherwise altered to reduce intensity of ambient lighttransmitted through that portion. As the user's head moves to the leftto a second position, the ambient light source may now appear to bealigned with a portion of the display lens 106 that is to the right ofthe left eye pupil. Accordingly, when the user's head is in the secondposition, the portion of the display lens 106 that is to the right ofthe left eye pupil may be darkened or otherwise altered to reduceintensity of ambient light transmitted through that portion to maintainthe user's visual experience. The portion of the display lens 106 thatis to the left of the left eye pupil that was previously darkened whenthe head was in the first position may be lightened or remain in thedarkened state. A sensor such as an outward facing camera that imagesthe field in front of the eyewear and that can provide mapping of thelocation of the objects including bright light sources in the field ofview of the sensor with respect to the lenses and the users eye, can beused to determine the portions of the lens that are to be altered, forexample, to attenuate light from bright objects the produce glare.Similarly, a database that includes a record of the location of objectsand, for example, their brightness, may also be used in determining theportion of the lens that is to be altered, for example, to attenuatelight from bright objects that produce glare. A head pose sensor and/orsystem may be used to determine the movement, position, and/ororientation of the head and/or body. This position may be used inconjunction with the database of locations of objects to determine theposition of the object with respect to the user's eye, the lens, and todetermine the portion(s) of the lens aligned with the object(s) as wellas the portion(s) of the lens that are to be altered.

FIG. 11 illustrates a flowchart 1100 that depicts a method of alteringambient light transmission characteristics through a display device thatwould improve a user's visual experience when using an embodiment of adisplay system 60. The method includes obtaining information regardingposition of various ambient light sources and/or glare producing objectsin a scene viewed by the user through the display device using one ormore sensors as shown in block 1105. For example, the one or more lightsensors 128, the outward facing camera(s) 112 and/or other sensorassemblies of the display system 60 can be configured to obtaininformation regarding the location and the nature of various ambientlight sources and/or glare producing objects in a scene viewed by theuser. The information obtained by the one or more light sensors 128, theoutward facing camera(s) 112 and/or other sensor assemblies of thedisplay system 60 can include the spectral characteristics of ambientlight and/or other characteristics of ambient light (e.g., intensity ofthe ambient light). As another example, one or more light sensors 128,the outward facing camera(s) 112 and/or other sensor assemblies of thedisplay system 60 can be configured to obtain information about thelocation of objects, areas, or regions of the forward field of view thatare bright and one or more areas or regions of the forward field of viewthat are dark. As yet another example, one or more light sensors 128,the outward facing camera(s) 112 and/or other sensor assemblies of thedisplay system 60 can be configured to obtain location of one or morebright ambient light sources and the intensity of light from the brightambient light sources and the intensity of light. The informationobtained by the one or more sensor assemblies is transmitted to one ormore electronic processing systems (e.g., the local processing & datamodule 140 and/or the remote processing module 150) for processing. Theelectronic processing systems can be local or remote. The one or moreelectronic processing systems can process the information obtained bythe one or more sensor assemblies and determine characteristics of theambient light at one or more locations of the display lens 106, as shownin block 1107. The determined characteristics can include the intensityof ambient light at one or more locations of the display lens 106 and/orthe spectral characteristics of ambient light at one or more locationsof the display lens 106. In some embodiments, the one or more electronicprocessing systems can also be configured to determine whether thesource of ambient light is the sun, a fluorescent light source, an LEDlight source, or a combination of these light sources. Additionally, asshown in block 1107, the one or more electronic processing systems canbe configured to identify portions of the display lens that appear to bealigned with the various ambient light sources and/or glare producingobjects in the scene viewed by the user through the display lens. Theone or more electronic processing systems can determine ambient lighttransmission characteristics at one or more locations of the displaylens 106 that will improve a user's visual experience based on thedetermined portions of the display lens 106 that coincide with thevarious ambient light sources and/or glare producing objects, theintensity and/or spectral characteristic of the various ambient lightsource and/or glare producing objects, as shown in block 1109.

For example, the one or more electronic processing systems can determinethe amount by which the one or more locations of the display lens 106should be darkened to improve user's visual experience. As anotherexample, based on the determination that ambient light is from a settingsun, the one or more electronic processing systems can determine thataltering the transmission characteristics of the portion of the displaylens 106 that is aligned with the sun as seen by the eye can reduceglare caused by the sun. Similarly, reducing the amount of light in oneor more wavelengths (e.g., red wavelengths) of the received light thatis transmitted through the portion of the display lens 106 that isaligned with the sun as seen by the eye can reduce glare and possiblyimprove user's visual experience.

The one or more electronic processing systems can send signals totrigger or cause one or more stimulus providing sources associated withthe display system 60 to alter the ambient light transmissioncharacteristics at one or more locations of the display lens 106 inaccordance with the determination made by the one or more electronicprocessing systems, as shown in block 1111. For example, the one or moreelectronic processing systems can send signals to turn on one or more ofthe optical, electrical, thermal and/or sonic/ultrasonic sourcesassociated with the display system 60 and provide an optical,electrical, thermal and/or sonic/ultrasonic signal to change thephysical/chemical characteristics of the molecules of the variableoptical material in at least a portion of the display lens 106 to alterthe light ambient transmission characteristic of that portion. Asanother example, the one or more electronic processing systems can sendsignals to turn on or otherwise cause an optical source or systemassociated with the display system 60 to provide an optical signal tochange the physical/chemical characteristics of the molecules of thevariable optical material in at least a portion of the display lens 106to alter the light ambient transmission characteristic of that portion.The optical signal can be of predetermined intensity and wavelength. Forexample, the optical signal can be a beam of visible or invisible lightof a certain wavelength. The molecules of the variable optical materialcan, for example, expand, shrink, move, twist or rotate in response tothe stimulus provided by the signal from the optical, electrical,thermal and/or sonic/ultrasonic sources associated with the displaysystem 60 and provide the desired ambient light altering characteristic(e.g., attenuation in one or more wavelength regions, light deflection,diffusion, etc.).

FIG. 12A schematically illustrates a side view of a display lens 1206disposed forward of a user's eye 1201. FIG. 12B schematicallyillustrates a front view of the display lens 1206 as seen from a sideopposite the eye side. FIG. 12C schematically illustrates a top view ofthe display lens 1206. An ambient light source 1210 in the scene viewedby the user through the display lens 1206 appears to be aligned with aregion 1207 of the display lens 1206. As illustrated in FIGS. 12A-12C,the ambient light source 1210 appears to be aligned with the region 1207of the display lens 1206 in both x- & y-directions. Similarly, theregion 1207 of the display lens 1206 appears to be aligned with thelight source 1210 as seen by the user's eye in both x- & y-directions.As discussed in this application, an electronic processing systemassociated with the display lens 1206 can be configured to alter/modifythe transmission of ambient light through the region 1207 of the displaylens 1206 to improve the user's visual experience. For example, in someembodiments, the region 1207 can be darkened as compared to otherportions of the display lens 1206 to reduce the intensity of ambientlight transmitted through that region. In some other embodiments,ambient light incident through the region 1207 may be directed away fromthe user's eye 1201. Other characteristics of the display may bealtered.

Various studies can be performed to characterize the light alteringcharacteristics of the variable optical material. Different studies canalso be performed to characterize the type of light alteration that willresult in a desired user experience for different types of ambient lightsources. For example, different embodiments of the display system 60 canbe tested prior to being used by a user to characterize the lightaltering characteristics of the variable optical material. The tests caninclude an analysis of the stimulus strength that would be required toachieve a certain alteration in a desired portion of the display 70 orthe display lens 106, the time interval between providing the stimulusand achieving the alteration in the desired portion of the display 70,the alteration that would provide an improved visual experience for anaverage user for different ambient light sources, etc. The results ofthe various studies can be stored in a database accessible by the localprocessing & data module 140 and/or the remote processing module 150.The local processing & data module 140 and/or the remote processingmodule 150 can access the results of the various studies whendetermining the nature of light altering capability of a certain portionof the display lens and the signals to send to various stimulusproviding sources.

In various embodiments, the display system 60 can be configured toobtain feedback from the user regarding the size and/or location of theportions of the display 70 and/or the display lens 106 that have alteredlight transmission capability and the extent to which the lighttransmission should be altered in various portions of the display 70and/or the display lens 106 to improve the user's visual experience. Insuch embodiments, the local processing & data module 140 and/or theremote processing module 150 can make an initial determination of thesize and/or location of the portions of the display 70 and/or thedisplay lens 106 that have altered light transmission capability basedon the information obtained from the various sensors and/or the imagingsystems associated with the system 60. The local processing & datamodule 140 and/or the remote processing module 150 can also make aninitial determination of the extent to which light transmission throughvarious portions of the display 70 and/or the display lens 106 should bealtered based on the results of the initial tests and studies. Thesystem 60 can then prompt the user using visual and/or aural signals andrequest feedback from the user regarding the size and/or location of theportions of the display 70 and/or the display lens 106 that have alteredambient light transmission and the extent to which light transmission isaltered through the various portions. The local processing & data module140 and/or the remote processing module 150 can adjust the size and/orlocation of the portions of the display 70 and/or the display lens 106that have altered light transmission capability and the extent to whichthe light transmission should be altered in various portions of thedisplay 70 and/or the display lens 106 based on feedback from the user.In this way, the visual experience can be improved based on a user'spreference. The user can provide feedback in a variety of ways. Forexample, the user can provide feedback using voice commands. As anotherexample, the user can use one or more buttons or knobs, a joystick, atouch pad or a track ball to provide feedback. As yet another example,the user can use gestures (e.g., hand gestures, facial gestures, blinkresponses, etc.) to provide feedback. An example of a display deviceconfigured to adjust size and/or location of the portions of the display70 and/or the display lens 106 that have altered light transmissioncapability and the extent to which the light transmission should bealtered in various portions of the display 70 and/or the display lens106 based on feedback from the user is discussed below.

Consider an embodiment of a display system that determines one or moreportions of the display lens that appear to be aligned with one or moreambient light sources in a scene viewed by the user through the displaylens. In response to the determination, the system can be configured todarken the one or more portions of the display lens that appear to bealigned with one or more ambient light sources in the scene. The systemcan then request feedback from the user regarding the size and/orlocations of the one or more darkened portions of the display lens andthe amount of darkening in those portions. The user can provide feedbackthat the system can use to adjust the size and/or locations of the oneor more darkened portions of the display lens and the amount ofdarkening in those portions.

The variable optical materials discussed herein can be configured to actas a filter that filter-out specific wavelengths of incoming light suchas, for example, blue light, red light, green light or some otherwavelength of light to enhance user experience. In various embodiments,the variable optical materials can be configured to direct incominglight towards or away from specific regions of the eye. In suchembodiment, the inward facing cameras 114 can be used to track movementsof the eye and the chemical/physical properties of the variable opticalmaterials can be controlled by providing stimulus from the system 60such that incoming light remains directed towards or away from specificregions of the eye despite movements of the eye. In various embodiments,the variable optical materials can be configured to partially orcompletely attenuate incoming light from an environment (e.g., toprevent sensory overload in certain environments).

Although attenuation, diffusion, refraction, redirection, filteringand/or scattering of ambient light through a portion of the display lens106 is discussed above, in any such case, in certain embodimentsdifferent lenses can attenuate, diffuse, refract, redirect, filterand/or scatter incident ambient light. For example, left and rightlenses can attenuate, diffuse, refract, redirect, filter and/or scatterincident ambient light by different amounts. Additionally differentportions of the left and right lenses can attenuate, diffuse, refract,redirect, filter and/or scatter incident ambient light differently.Direct control over the degree of attenuation and the portions of thelenses that are attenuated enables different portions of the left andright lenses that have different shapes and/or sizes to be attenuated aswell as different magnitudes and distributions of attenuation. Othercharacteristics such as spectral characteristics of the left and rightlenses and the attenuation thereof can be different.

It is contemplated that various embodiments may be implemented in orassociated with a variety of applications such as imaging systems anddevices, display systems and devices, spatial light modulators, liquidcrystal based devices, polarizers, wave guide plates, etc. Thestructures, devices and methods described herein may particularly finduse in displays such as wearable displays (e.g., head mounted displays)that can be used for augmented and/or virtually reality. More generally,the described embodiments may be implemented in any device, apparatus,or system that can be configured to display an image, whether in motion(such as video) or stationary (such as still images), and whethertextual, graphical or pictorial. It is contemplated, however, that thedescribed embodiments may be included in or associated with a variety ofelectronic devices such as, but not limited to: mobile telephones,multimedia Internet enabled cellular telephones, mobile televisionreceivers, wireless devices, smartphones, Bluetooth® devices, personaldata assistants (PDAs), wireless electronic mail receivers, hand-held orportable computers, netbooks, notebooks, smartbooks, tablets, printers,copiers, scanners, facsimile devices, global positioning system (GPS)receivers/navigators, cameras, digital media players (such as MP3players), camcorders, game consoles, wrist watches, clocks, calculators,television monitors, flat panel displays, electronic reading devices(e.g., e-readers), computer monitors, auto displays (including odometerand speedometer displays, etc.), cockpit controls and/or displays,camera view displays (such as the display of a rear view camera in avehicle), electronic photographs, electronic billboards or signs,projectors, architectural structures, microwaves, refrigerators, stereosystems, cassette recorders or players, DVD players, CD players, VCRs,radios, portable memory chips, washers, dryers, washer/dryers, parkingmeters, head mounted displays and a variety of imaging systems. Thus,the teachings are not intended to be limited to the embodiments depictedsolely in the Figures, but instead have wide applicability as will bereadily apparent to one having ordinary skill in the art.

Various modifications to the embodiments described in this disclosuremay be readily apparent to those skilled in the art, and the genericprinciples defined herein may be applied to other embodiments withoutdeparting from the spirit or scope of this disclosure. Various changesmay be made to the invention described and equivalents may besubstituted without departing from the true spirit and scope of theinvention. In addition, many modifications may be made to adapt aparticular situation, material, composition of matter, process, processact(s) or step(s) to the objective(s), spirit or scope of the presentinvention. All such modifications are intended to be within the scope ofclaims associated with this disclosure.

The word “exemplary” is used exclusively herein to mean “serving as anexample, instance, or illustration.” Any embodiment described herein as“exemplary” is not necessarily to be construed as preferred oradvantageous over other embodiments. Additionally, a person havingordinary skill in the art will readily appreciate, the terms “upper” and“lower”, “above” and “below”, etc., are sometimes used for ease ofdescribing the figures, and indicate relative positions corresponding tothe orientation of the figure on a properly oriented page, and may notreflect the orientation of the structures described herein, as thosestructures are implemented.

Certain features that are described in this specification in the contextof separate embodiments also can be implemented in combination in asingle embodiment. Conversely, various features that are described inthe context of a single embodiment also can be implemented in multipleembodiments separately or in any suitable subcombination. Moreover,although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. Further, the drawings may schematically depict one more exampleprocesses in the form of a flow diagram. However, other operations thatare not depicted can be incorporated in the example processes that areschematically illustrated. For example, one or more additionaloperations can be performed before, after, simultaneously, or betweenany of the illustrated operations. In certain circumstances,multitasking and parallel processing may be advantageous. Moreover, theseparation of various system components in the embodiments describedabove should not be understood as requiring such separation in allembodiments, and it should be understood that the described programcomponents and systems can generally be integrated together in a singlesoftware product or packaged into multiple software products.Additionally, other embodiments are within the scope of the followingclaims. In some cases, the actions recited in the claims can beperformed in a different order and still achieve desirable results.

The invention includes methods that may be performed using the subjectdevices. The methods may comprise the act of providing such a suitabledevice. Such provision may be performed by the end user. In other words,the “providing” act merely requires the end user obtain, access,approach, position, set-up, activate, power-up or otherwise act toprovide the requisite device in the subject method. Methods recitedherein may be carried out in any order of the recited events which islogically possible, as well as in the recited order of events.

Example aspects of the invention, together with details regardingmaterial selection and manufacture have been set forth above. As forother details of the present invention, these may be appreciated inconnection with the above-referenced patents and publications as well asgenerally known or appreciated by those with skill in the art. The samemay hold true with respect to method-based aspects of the invention interms of additional acts as commonly or logically employed.

In addition, though the invention has been described in reference toseveral examples optionally incorporating various features, theinvention is not to be limited to that which is described or indicatedas contemplated with respect to each variation of the invention. Variouschanges may be made to the invention described and equivalents (whetherrecited herein or not included for the sake of some brevity) may besubstituted without departing from the true spirit and scope of theinvention. In addition, where a range of values is provided, it isunderstood that every intervening value, between the upper and lowerlimit of that range and any other stated or intervening value in thatstated range, is encompassed within the invention.

Also, it is contemplated that any optional feature of the inventivevariations described may be set forth and claimed independently, or incombination with any one or more of the features described herein.Reference to a singular item, includes the possibility that there areplural of the same items present. More specifically, as used herein andin claims associated hereto, the singular forms “a,” “an,” “said,” and“the” include plural referents unless the specifically stated otherwise.In other words, use of the articles allow for “at least one” of thesubject item in the description above as well as claims associated withthis disclosure. It is further noted that such claims may be drafted toexclude any optional element. As such, this statement is intended toserve as antecedent basis for use of such exclusive terminology as“solely,” “only” and the like in connection with the recitation of claimelements, or use of a “negative” limitation.

Without the use of such exclusive terminology, the term “comprising” inclaims associated with this disclosure shall allow for the inclusion ofany additional element—irrespective of whether a given number ofelements are enumerated in such claims, or the addition of a featurecould be regarded as transforming the nature of an element set forth insuch claims. Except as specifically defined herein, all technical andscientific terms used herein are to be given as broad a commonlyunderstood meaning as possible while maintaining claim validity.

The breadth of the present invention is not to be limited to theexamples provided and/or the subject specification, but rather only bythe scope of claim language associated with this disclosure.

What is claimed is:
 1. A user-wearable display device comprising: aframe configured to mount on the user; an augmented reality displayattached to the frame and configured to direct images to an eye of theuser and to provide a first field of view of an environment surroundingthe user; one or more sensors configured to obtain information about anambient light condition in the environment surrounding the user, atleast one of the one or more sensors being an image capture device witha second field of view of the environment surrounding the user, whereinthe second field of view is offset from the first field of view; avariable optical material that undergoes a physical and/or a chemicalchange in response to a stimulus; a source configured to provide thestimulus; and processing electronics configured to: trigger the sourceto provide the stimulus to the variable optical material to effect aphysical and/or a chemical change in the material at a location selectedbased on the information obtained by the one or more sensors and on arelationship between the first field of view of the augmented realitydisplay and the second field of view of the image capture device suchthat at least one of intensity of ambient light, spectral content ofambient light, or direction of ambient light is changed.
 2. Theuser-wearable device of claim 1, wherein the augmented reality displaycomprises a waveguide configured to: allow a view of the environmentsurrounding the user through the waveguide; and form images by directinglight out of the waveguide and into an eye of the user.
 3. Theuser-wearable device of claim 2, wherein the waveguide is part of astack of waveguides, wherein each waveguide of the stack is configuredto output light with different amounts of divergence in comparison toone or more other waveguides of the stack of waveguides.
 4. Theuser-wearable device of claim 1, wherein the one or more sensors furthercomprise a global positioning sub-system or an environmental sensor. 5.The user-wearable device of claim 1, further comprising another imagecapture device configured to track movement of eyes of the user.
 6. Theuser-wearable device of claim 1, further comprising a light sourceconfigured to generate a projection beam based on data associated withthe images directed to the eye of the user.
 7. The user-wearable deviceof claim 1, wherein the source comprises an optical source configured todirect visible or invisible light to one or more portions of thedisplay.
 8. The user-wearable device of claim 1, wherein the sourcecomprises an electrical source configured to provide an electricalsignal to one or more portions of the display.
 9. The user-wearabledevice of claim 1, wherein the source comprises a thermal sourceconfigured to provide a thermal radiation to one or more portions of thedisplay.
 10. The user-wearable device of claim 1, wherein the sourcecomprises a sonic/ultrasonic system configured to providesonic/ultrasonic energy to one or more portions of the display.
 11. Theuser-wearable device of claim 1, wherein the variable optical materialis embedded in a surface of the display.
 12. The user-wearable device ofclaim 1, wherein the variable optical material is disposed over asurface of the display.
 13. The user-wearable device of claim 1, whereinthe variable optical material includes organic or inorganic compounds.14. The user-wearable device of claim 1, wherein the variable opticalmaterial comprises electroactive proteins.
 15. The user-wearable deviceof claim 1, wherein the variable optical material comprises moleculesthat exhibit a change in size or shape in response to the stimulus. 16.The user-wearable device of claim 1, wherein the variable opticalmaterial comprises molecules that move, rotate, twist, or shift inresponse to the stimulus.
 17. The user-wearable device of claim 1,wherein the variable optical material comprises molecules that movetogether and/or adhere together in response to the stimulus.
 18. Theuser-wearable device of claim 1, wherein the variable light opticalmaterial comprises molecules that move away from each other in responseto the stimulus.
 19. The user-wearable device of claim 1, wherein thevariable optical material comprises molecules that form nanostructuresin response to the stimulus.
 20. The user-wearable device of claim 1,wherein the display comprises a first ocular region corresponding to afirst eye of the user and a second ocular region corresponding to asecond eye of the user, and wherein the processing electronics areconfigured to trigger the source to provide the stimulus to a portion ofthe display to effect a physical and/or a chemical change in thevariable optical material based on the information obtained by the oneor more sensors such that at least one of intensity of ambient light,spectral content of ambient light, or direction of ambient light ischanged through the first ocular region as a result of stimulus from asource triggered by the processing electronics.
 21. The user-wearabledevice of claim 1, wherein the display comprises a first ocular regioncorresponding to a first eye of the user and a second ocular regioncorresponding to a second eye of the user, and wherein the processingelectronics are configured to trigger the source to provide the stimulusto a portion of the display to effect a physical and/or a chemicalchange in the material based on the information obtained by the one ormore sensors such that at least one of intensity of ambient light,spectral content of ambient light, or direction of ambient light throughthe first ocular region is changed differently as compared to intensityof ambient light, spectral content of ambient light, or direction ofambient light through the second ocular region.
 22. The user-wearabledevice of claim 1, wherein the processing electronics are configured totrigger the source to provide the stimulus to the display to effect aphysical and/or a chemical change in the material based on theinformation obtained by the one or more sensors such that attenuation ofintensity of ambient light transmitted through a first portion of thedisplay is greater than attenuation of intensity of ambient lighttransmitted through a second portion of the display.
 23. Theuser-wearable device of claim 22, wherein the intensity of ambient lightincident on the first portion of the display is greater than intensityof ambient light incident on the second portion of the display.
 24. Theuser-wearable device of claim 22, wherein the processing electronics areconfigured to trigger the source to provide the stimulus to the displayto effect a physical and/or a chemical change in the material based onthe information obtained by the one or more sensors such that theintensity of ambient light transmitted through the second portion of thedisplay is reduced.
 25. The user-wearable device of claim 1, wherein thedisplay comprises a first ocular region corresponding to a first eye ofthe user and a second ocular region corresponding to a second eye of theuser, and wherein the processing electronics are configured to triggerthe source to provide the stimulus to the display to effect a physicaland/or a chemical change in the material based on the informationobtained by the one or more sensors such that intensity of ambient lighttransmitted through a portion of the first ocular region is reduced. 26.The user-wearable device of claim 1, wherein the processing electronicsare configured to trigger the source to provide the stimulus to thedisplay to effect a physical and/or a chemical change in the materialbased on the information obtained by the one or more sensors such thatthe spectrum of ambient light transmitted through a first portion of thedisplay is different than the spectrum of ambient light transmittedthrough a second portion of the display.
 27. The user-wearable device ofclaim 1, wherein the display comprises a first lens corresponding to afirst eye of the user and a second lens corresponding to a second eye ofthe user, and wherein the processing electronics are configured totrigger the source to provide the stimulus to the display to effect aphysical and/or a chemical change in the variable optical materialassociated with the first lens based on the information obtained by theone or more sensors such that intensity of ambient light transmittedthrough only the first lens is reduced as a result of stimulus from asource triggered by the processing electronics.
 28. The user-wearabledevice of claim 1, wherein the display comprises a first lenscorresponding to a first eye of the user and a second lens correspondingto a second eye of the user, and wherein the processing electronics areconfigured to trigger the source to provide the stimulus to the displayto effect a physical and/or a chemical change in the variable opticalmaterial associated with the first lens based on the informationobtained by the one or more sensors such that intensity of ambient lighttransmitted through a portion of the first lens is reduced by an amountgreater than another portion of the first lens.
 29. The user-wearabledevice of claim 28, wherein the processing electronics are configured totrigger the source to provide the stimulus to the display to effect aphysical and/or a chemical change in the variable optical materialassociated with the second lens based on the information obtained by theone or more sensors such that intensity of ambient light transmittedthrough a portion of the second lens is reduced.
 30. The user-wearabledevice of claim 1, wherein the display comprises a first lenscorresponding to a first eye of the user and a second lens correspondingto a second eye of the user, and wherein the processing electronics areconfigured to trigger the source to provide the stimulus to the displayto effect a physical and/or a chemical change in the variable opticalmaterial associated with the first lens based on the informationobtained by the one or more sensors such that intensity of ambient lighttransmitted through the first lens is attenuated more than through thesecond lens.
 31. The user-wearable device of claim 30, wherein theprocessing electronics are configured to trigger the source to providethe stimulus to the display to effect a physical and/or a chemicalchange in the variable optical material associated with the second lensbased on the information obtained by the one or more sensors such thatintensity of ambient light transmitted through the second lens isreduced.
 32. The user-wearable device of claim 1, wherein the displaycomprises a first lens corresponding to a first eye of the user and asecond lens corresponding to a second eye of the user, and wherein theprocessing electronics are configured to trigger the source to providethe stimulus to the display to effect a physical and/or a chemicalchange in variable optical material associated with the first or secondlens based on the information obtained by the one or more sensors suchthat spectrum of ambient light transmitted through the first and secondlenses is different.
 33. The user-wearable device of claim 1, whereinthe display comprises a first lens corresponding to a first eye of theuser and a second lens corresponding to a second eye of the user, andwherein the processing electronics are configured to trigger the sourceto provide the stimulus to the display to effect a physical and/or achemical change in the variable optical material associated with thefirst or second lens based on the information obtained by the one ormore sensors such that the spectrum of ambient light transmitted througha portion of the first lenses is different than another portion of thefirst lens.
 34. The user-wearable device of claim 33, wherein thedisplay comprises a first lens corresponding to a first eye of the userand a second lens corresponding to a second eye of the user, and whereinthe processing electronics are configured to trigger the source toprovide the stimulus to the display to effect a physical and/or achemical change in the variable optical material associated with thefirst or second lens based on the information obtained by the one ormore sensors such that the spectrum of ambient light transmitted througha portion of the first lenses is different than another portion of thesecond lens.
 35. The user-wearable device of claim 1, wherein an objectas seen by the wearer's eye through the display appears to be alignedwith at least one portion of the display, and wherein the processingelectronics are configured to cause the source to provide the stimulusto the at least one portion of the display for which the object appearsto be aligned to effect a physical and/or a chemical change in thevariable optical material such that at least one of intensity of lightfrom said object, spectral content of said light from said object ordirection of said light from said object is changed.
 36. Theuser-wearable device of claim 35, wherein the processing electronics areconfigured to determine the at least one portion of the display forwhich the object appears to be aligned based on the movement of theuser's head as tracked by said one or more sensors.
 37. Theuser-wearable device of claim 35, wherein the processing electronics areconfigured to cause the source to provide the stimulus to the at leastone portion of the display to effect a physical and/or a chemical changein the variable optical material such that the intensity of ambientlight reduced.
 38. The user-wearable device of claim 1, furthercomprising a head pose sensor.
 39. The user-wearable device of claim 1,further configured to adjust the location of the at least one portion ofthe display through which at least one of intensity of ambient light,spectral content of ambient light, or direction of ambient light ischanged based on feedback from the user.
 40. The user-wearable device ofclaim 1, further configured to adjust the size of the at least oneportion of the display through which at least one of intensity ofambient light, spectral content of ambient light, or direction ofambient light is changed based on feedback from the user.
 41. Theuser-wearable device of claim 1, further configured to adjust the amountby which at least one of intensity of ambient light, spectral content ofambient light, or direction of ambient light is changed based onfeedback from the user.
 42. The user-wearable device of claim 1, whereinthe variable optical material is distributed unevenly across a surfaceof the display.
 43. The user-wearable device of claim 42, wherein afirst portion of the display has a higher density of the variableoptical material than a second portion of the display.
 44. Theuser-wearable device of claim 1, wherein the variable optical materialcomprises a first layer of a first variable optical material and asecond layer of a second variable optical material.
 45. Theuser-wearable device of claim 44, wherein the first variable opticalmaterial acts on different wavelengths of the ambient light than thesecond variable optical material.
 46. The user-wearable device of claim1, wherein the variable optical material is provided in an add-on devicethat can be removably attached to the display.
 47. The user-wearabledevice of claim 1, wherein the device is configured to: determine theuser's position in the environment; and provide the stimulus based onthe user's position in the environment.
 48. The user-wearable device ofclaim 47, wherein the device is further configured to: store one or morelocations of ambient light sources in the environment; and provide thestimulus based on the user's position relative to the one or more storedlocations of ambient light sources.
 49. The user-wearable device ofclaim 48, wherein the device is further configured to provide thestimulus based on the time of day.
 50. The user-wearable device of claim1, wherein the variable optical material is configured to diffract,refract, or diffuse the ambient light.
 51. The user-wearable device ofclaim 1, wherein the device is configured to select the location for thestimulus based on a relatively bright source of ambient light detectedby the one or more sensors, wherein the relatively bright source ofambient light is determined by dividing the one or more sensors into aplurality of pixel groups and determining an average brightness for eachof the pixel groups.
 52. A method of manipulating light transmittedthrough a user-wearable display device with a first field of view of anenvironment surrounding the user, the display device comprising adisplay surface including a variable optical material that varies atleast one of intensity of ambient light, spectral content of ambientlight, or direction of ambient light transmitted through the displaysurface in response to a stimulus, the method comprising: obtaining oneor more measurements about an ambient light condition in the environmentsurrounding the user using one or more sensors, at least one of the oneor more sensors being an image capture device with a second field ofview of the environment surrounding the user, wherein the second fieldof view is offset from the first field of view; and controlling a sourceto provide a stimulus to the display surface to effect a physical and/orchemical change in the material at a location selected based on the oneor more measurements obtained by the one or more sensors and on arelationship between the first field of view of the display device andthe second field of view of the image capture device such that at leastone of intensity of ambient light, spectral content of ambient light, ordirection of ambient light transmitted through the display surface ischanged.